Issue Details (XML | Word | Printable)

Key: DERBY-557
Type: Bug Bug
Status: Closed Closed
Resolution: Fixed
Priority: Minor Minor
Assignee: Knut Anders Hatlen
Reporter: Knut Anders Hatlen
Votes: 0
Watchers: 0
Operations

If you were logged in you would be able to see more operations.
Derby

Client driver gets OutOfMemoryError when re-executing statement without closing ResultSet

Created: 09/Sep/05 12:31 AM   Updated: 15/Feb/06 10:11 PM
Return to search
Component/s: Network Client
Affects Version/s: 10.1.1.0, 10.1.2.1, 10.2.1.6
Fix Version/s: 10.1.2.1, 10.2.1.6

Time Tracking:
Not Specified

File Attachments:
  Size
Text File Licensed for inclusion in ASF works DERBY-557-regression-test-v2.diff 2006-02-12 09:11 AM Knut Anders Hatlen 3 kB
File Licensed for inclusion in ASF works DERBY-557-regression-test-v3.diff 2006-02-15 06:49 PM Knut Anders Hatlen 3 kB
Text File Licensed for inclusion in ASF works DERBY-557-regression-test.diff 2006-02-12 07:47 AM Knut Anders Hatlen 3 kB
Text File Licensed for inclusion in ASF works DERBY-557-regression-test.stat 2006-02-12 07:47 AM Knut Anders Hatlen 0.3 kB
File Licensed for inclusion in ASF works DERBY-557.diff 2005-09-12 06:44 PM Knut Anders Hatlen 4 kB
Text File Licensed for inclusion in ASF works DERBY-557.stat 2005-09-12 06:44 PM Knut Anders Hatlen 0.2 kB
Text File Licensed for inclusion in ASF works derbyall_report.txt 2005-09-12 06:44 PM Knut Anders Hatlen 16 kB

Resolution Date: 15/Feb/06 09:50 PM


 Description  « Hide
When re-executing a statement without closing the old ResultSet, some of the old ResultSet's resources are not released, and the network client will eventually get an OutOfMemoryError.

Example:

Connection c = DriverManager.getConnection("jdbc:derby://localhost/mydb");
PreparedStatement ps = c.prepareStatement("select * from sys.systables");
while (true) {
ResultSet rs = ps.executeQuery();
}

This code will run for some time and then throw an OutOfMemoryError. Same thing happens with Statement instead of PreparedStatement. If rs.close() is added in the loop, the code works. Explicitly closing the ResultSet should not be necessary according to this quote from the JDBC 3.0 spec:

  For Select statements, the statement is complete when the associated result set is closed. The result set is closed as soon as one of the following occurs:
     - all of the rows have been retrieved
     - the associated Statement object is re-executed
     - another Statement object is executed on the same connection

 All   Comments   Work Log   Change History   Subversion Commits      Sort Order: Ascending order - Click to sort in descending order
Repository Revision Date User Message
ASF #289539 Fri Sep 16 13:52:14 UTC 2005 kmarsden DERBY-557
Client driver gets OutOfMemoryError when re-executing statement without closing ResultSet

Attached a patch which fixes the bug. The patch ensures that the ResultSets associated with a Statement/PreparedStatement are removed from CommitAndRollbackListeners_ in org.apache.derby.client.am.Connection when the statement is re-executed.

I have run derbyall with only one error (wrong length of encryption key, not related to the patch). I have also run the program in the problem description (both with Statement and PreparedStatement), and the memory usage doesn't increase over time.

Contributed by Knut Anders Hatlen
Files Changed
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/Statement.java
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/ResultSet.java
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/PreparedStatement.java

Repository Revision Date User Message
ASF #306950 Thu Oct 06 22:55:33 UTC 2005 kmarsden DERBY-557
Client driver gets OutOfMemoryError when re-executing statement without closing ResultSet
Merged from trunk with
 svn merge -r 289538:289539 https://svn.apache.org/repos/asf/db/derby/code/trunk

Contributed by Knut Anders Hatlen
Files Changed
MODIFY /db/derby/code/branches/10.1/java/client/org/apache/derby/client/am/ResultSet.java
MODIFY /db/derby/code/branches/10.1/java/client/org/apache/derby/client/am/PreparedStatement.java
MODIFY /db/derby/code/branches/10.1/java/client/org/apache/derby/client/am/Statement.java

Repository Revision Date User Message
ASF #369612 Tue Jan 17 00:10:58 UTC 2006 kmarsden DERBY-210 - Network Server will leak prepared statements if not explicitly closed by the user until the connection is closed


 I am uploading a combined patch 'DERBY-210.diff' which solves the memory leak. As Bryan suggested, I am uploading this patch and will open jira issues for other optimizations. Patch does the following:

* Eliminates the below references to PreparedStatement objects by using WeakHashMap instead of LinkedList. When there are no other references to the keys in a WeakHashMap, they will get removed from the map and can thus get garbage-collected. They do not have to wait till the Connection object is collected.
       - 'openStatements_' in org.apache.derby.client.am.Connection
       - 'CommitAndRollbackListeners_' in org.apache.derby.client.am.Connection

* Removes the list 'RollbackOnlyListeners_' since this is not being used.

* Updates the following comment for openStatements_:
// Since DERBY prepared statements must be re-prepared after a commit,
// then we must traverse this list after a commit and notify statements
// that they are now in an un-prepared state.
final java.util.LinkedList openStatements_ = new java.util.LinkedList();

In the code, I did not see this list being traversed after a commit to re-prepare statements. Also, I think this is not needed since Derby does not require re-prepare of statements after a commit. Currently, this list is used to close all open statements when the originating connection is closed.

* Removes all ResultSets from HashTable 'positionedUpdateCursorNameToResultSet_' in SectionManager. Only result sets of positioned update statements were being removed from this hashtable whereas all result sets were added. Because of this, client driver was holding on to result sets and statements even after rs.close() was called.

* Adds a test 'derbyStress.java' to jdbcapi suite. This test is based on the repro for this patch. Without this patch, it fails when run with client driver. Kathey had suggested in another mail that tests for client memory leak problems (DERBY-557, DERBY-210) can be added to same test. I did not see an existing test. So I created this new test. If DERBY-557 does not have a test, I think it can be added to this new test.

* Excludes the new test from running with jcc because jcc gives out of memory error.

* Creates 'derbyStress_app.properties' with following property 'jvmflags=-Xmx64M' to guarantee the test fails on all machines.

Successfully ran derbyall with Sun JDK 1.4.2 on Windows XP. Please take a look at this patch.


Contributed by Deepa Remesh
Files Changed
ADD /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/derbyStress_app.properties
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/Connection.java
ADD /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/derbyStress.java
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/Statement.java
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/Lob.java
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/DerbyNet.exclude
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/jdbcapi.runall
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/ResultSet.java
ADD /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/derbyStress.out
MODIFY /db/derby/code/trunk/java/client/org/apache/derby/client/am/PreparedStatement.java

Repository Revision Date User Message
ASF #377998 Wed Feb 15 12:47:40 UTC 2006 bernt DERBY-557 Added a test case for DERBY-557. Submitted by Knut Anders Hatlen
Files Changed
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/suites/DerbyNetClient.exclude
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/tests/jdbcapi/derbyStress.java
MODIFY /db/derby/code/trunk/java/testing/org/apache/derbyTesting/functionTests/master/derbyStress.out