Closing a resultset after retrieving a large > 32K value with Network Server does not release locks

• Key: DERBY-255
• Type: Bug
• Status: Closed
• Resolution: Fixed
• Priority: Major
• Assignee: Kathey Marsden
• Reporter: Kathey Marsden
• Component/s: Network Server
• Affects Version/s: 10.0.2.0, 10.0.2.1, 10.0.2.2, 10.1.1.0
• Fix Version/s: 10.1.1.0

Description

Closing a resultset after retriving BLOB or CLOB data > 32K, does not release locks properly. Network Server uses getClob, getBlob to retrieve the data even if the application uses getCharacteStream, etc, so holds locks to the end of the transaction.


To reproduce run attached repro
java LargeDataLocks derbynetclient
 
To see the difference with embedded
java LargeDataLocks derby

Comments

[Kathey Marsden]

repro for setCharacterStream holding locks after retrieving BLOB/CLOB data greater than 32K

[Kathey Marsden]

This fixes DERBY-255

Closing a resultset after retriving BLOB or CLOB data > 32K, does not release locks properly.

Network server/client materializes the LOB on the client and cannot differentiate getBlob from getBytes or getBinaryStream. Previously, network server would always call getBlob/getClob for any lob related call. This change changes network server to use getBytes/getString and not hold locks for any of the calls.

The implementation adds a new class EXTDTAInputStream to network server to localize the stream handling for large objects. This should make it easier to adjust in the future as improvements are made in the large object handling. Because we need a length in order to write a stream, EXTDTAInputStream currently call getBytes or getString to get the length and stream out that object. This is apparently required because we cannot reset the input stream after traversing it to get the length.


Future suggestions for changes to network server to handle this in a more complete way would be to:

1) Change DDMWriter.writeScalarStream to not require a length and optimize EXTDTAObjectInputStream accordingly

2) Add support for lob locators with network server. The getBlob, getClob calls would use the locators and would hold locks until the end of the transaction.

[Kathey Marsden]

Checked this in with revision 179014.

Sending java\drda\org\apache\derby\impl\drda\DDMWriter.java
Sending java\drda\org\apache\derby\impl\drda\DRDAConnThread.java
Adding java\drda\org\apache\derby\impl\drda\EXTDTAInputStream.java
Sending java\testing\org\apache\derbyTesting\functionTests\master\DerbyNet\blobclob4BLOB.out
Sending java\testing\org\apache\derbyTesting\functionTests\master\DerbyNetClient\blobclob4BLOB.out
Sending java\testing\org\apache\derbyTesting\functionTests\master\blobclob4BLOB.out
Sending java\testing\org\apache\derbyTesting\functionTests\tests\jdbcapi\blobclob4BLOB.java
Transmitting file data .......
Committed revision 179014.

[Kathey Marsden]

Checked in svn 179017 + 179198 to resolve this issue.

[Kathey Marsden]

Sunitha and I chatted on IRC about DERBY-255. First transcript was lost but recapped in the second.

* Since we cannot traverse the stream twice to get length and then data Network Server currently does getBytes for Blob calls and needs sufficient JVM memory allocated.
* Real solution is
   1) DERBY-326 to enable streaming without the length.
   2) DERBY-327 to use lob locators for performance and differentiation of client calls
      to getBlob vs getBinaryStream for locking.
* As an aside noted that the embedded Blob.length() call could probably use available(), skip() to improve performance instead of reading the data.