[DERBY-326] Improve streaming of large objects for network server and client

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 10.1.2.1
• Fix Version/s: 10.2.1.6
• Component/s: Network Client, Network Server
• Bug behavior facts: Performance
• Assignee: Tomohito Nakayama
• Reporter: Katherine Marsden
• Created: 31/May/05 19:38
• Updated: 14/Aug/09 16:34
• Resolved: 29/Apr/09 01:01

Description

Currently the stream writing methods in network server and client require a length parameter. This means that we have to get the length of the stream before sending it. For example in network server in EXTDTAInputStream we have to use getString and getbytes() instead of getCharacterStream and getBinaryStream so that we can get the length.

SQLAM Level 7 provides for the enhanced LOB processing to allow streaming without indicating the length, so, the writeScalarStream methods in
network server DDMWriter.java and network client Request.java can be changed to not require a length.

Code inspection of these methods seems to indicate that while the length is never written it is used heavily in generating the DSS. One strange thing is that it appears on error, the stream is padded out to full length with zeros, but an actual exception is never sent. Basically I think perhaps these methods need to be rewritten from scratch based on the spec requirements for lobs.

After the writeScalarStream methods have been changed, then EXTDAInputStream can be changed to properly stream LOBS. See TODO tags in this file for more info. I am guessing similar optimizations available in the client as well, but am not sure where that code is.

Issue Links

• incorporates: DERBY-1302 Indent source files touched in DERBY-326
• incorporates: DERBY-1303 The test of derbynet/SuicideOfStreaming.java is failed when programs are build as Insane.
• incorporates: DERBY-1301 Improve streaming of large objects for network server and client,from client to server
• is blocked by: DERBY-721 State of InputStream retrieved from resultset is not clean , if there exists previous InputStream .
• relates to: DERBY-2892 Closing a resultset after retrieving a large > 32665 bytes value with Network Server does not release locks
• relates to: DERBY-550 BLOB : java.lang.OutOfMemoryError with network JDBC driver (org.apache.derby.jdbc.ClientDriver)
• relates to: DERBY-4350 javaException is created but not thrown or logged in DRDAConnThread.closeStream(InputStream stream)
• relates to: DERBY-760 Implement routines that will write/read data from a byte[] in drda format rather than calling getXXX and setXXX methods
• relates to: DERBY-917 Disable largedata/LobLimits.java test from running in network server mode

Sub-Tasks

• Fix how to measure performance of lob streaming: Closed
• Add description of derby.drda.streamOutBufferSize to "Derby Server and Administration Guide": Closed
• Indent source files touched in DERBY-326: Closed