Derby
  1. Derby
  2. DERBY-5336

Repeated database creation causes OutOfMemoryError

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.8.1.2
    • Fix Version/s: 10.8.2.2, 10.9.1.0
    • Component/s: Services
    • Labels:
      None
    • Environment:
      Windows 7 64-bit, jdk1.6.0_26
    • Urgency:
      Normal
    • Issue & fix info:
      Repro attached, Workaround attached
    • Bug behavior facts:
      Crash, Regression

      Description

      Repeatedly creating, querying, updating, and shutting down a database or databases eventually causes OutOfMemoryError.

      Java source reproducing the problem is attached.

      1. derby-5336-2a-regression_test.diff
        5 kB
        Kristian Waagan
      2. LicensedRepro.java
        3 kB
        Aja Walker
      3. derby-5336-1a-remove_context_on_stop.diff
        0.8 kB
        Kristian Waagan
      4. DerbyOutOfMemoryErrorRepro.java
        3 kB
        Aja Walker

        Issue Links

          Activity

          Hide
          Aja Walker added a comment - - edited

          Attached source code for reproducing the error.

          Show
          Aja Walker added a comment - - edited Attached source code for reproducing the error.
          Hide
          Knut Anders Hatlen added a comment -

          I'm able to reproduce the OOME on trunk. It looks like the objects belonging to the stopped database instances are still referenced from the index statistics update daemon and cannot be freed.

          Disabling automatic update of index statistics seems to work around the problem. If I run the repro with -Dderby.storage.indexStats.auto=false, I don't see the OOME.

          I'm marking this bug as a regression, since the OOME is not seen with 10.7.1.1.

          Show
          Knut Anders Hatlen added a comment - I'm able to reproduce the OOME on trunk. It looks like the objects belonging to the stopped database instances are still referenced from the index statistics update daemon and cannot be freed. Disabling automatic update of index statistics seems to work around the problem. If I run the repro with -Dderby.storage.indexStats.auto=false, I don't see the OOME. I'm marking this bug as a regression, since the OOME is not seen with 10.7.1.1.
          Hide
          Kristian Waagan added a comment -

          Seems the context used by the istat deamon isn't removed when the database is shut down.
          I'm running regression tests on a possible fix.

          On a related note, I'm wondering if the context class loader should be set to null for the istat daemon thread?

          Show
          Kristian Waagan added a comment - Seems the context used by the istat deamon isn't removed when the database is shut down. I'm running regression tests on a possible fix. On a related note, I'm wondering if the context class loader should be set to null for the istat daemon thread?
          Hide
          Kristian Waagan added a comment -

          Aja,

          I see that you didn't tick the grant license radio button when uploading the repro.
          Do you mind doing that, such that I can copy most of your repro code to add a regression test for this bug?

          Of course, you can also modify the repro to fit the Derby test framework yourself if you want to (the test should go into the low memory suite)

          Show
          Kristian Waagan added a comment - Aja, I see that you didn't tick the grant license radio button when uploading the repro. Do you mind doing that, such that I can copy most of your repro code to add a regression test for this bug? Of course, you can also modify the repro to fit the Derby test framework yourself if you want to (the test should go into the low memory suite)
          Hide
          Kristian Waagan added a comment -

          Attached patch 1a, which removes the istat daemon context by invoking the cleanup code in the system context.

          Tests passed on Linux and Solaris 11 using Java 6.
          Patch ready for review.

          Show
          Kristian Waagan added a comment - Attached patch 1a, which removes the istat daemon context by invoking the cleanup code in the system context. Tests passed on Linux and Solaris 11 using Java 6. Patch ready for review.
          Hide
          Aja Walker added a comment -

          Attaching the repro source with the license box ticked this time.

          Show
          Aja Walker added a comment - Attaching the repro source with the license box ticked this time.
          Hide
          Kristian Waagan added a comment -

          Thanks, Aja.

          Attaching patch 2a, where I copied Aja's repro and modified it to fit with the Derby test framework (JUnit based):

          • switched from DriverManager to DataSource
          • replaced commons io call to delete directory
          • added an extra sanity check
          • added test as testRepeatedDatabaseCreationWithAutoStats in memory.MemoryLeakFixesTest. Should be run with 'ant junit-lowmem' (after building Derby), which uses a 16 MB heap.

          The test failed when run without the fix, it succeeded with patch 1a applied.
          Patch ready for review.

          Show
          Kristian Waagan added a comment - Thanks, Aja. Attaching patch 2a, where I copied Aja's repro and modified it to fit with the Derby test framework (JUnit based): switched from DriverManager to DataSource replaced commons io call to delete directory added an extra sanity check added test as testRepeatedDatabaseCreationWithAutoStats in memory.MemoryLeakFixesTest. Should be run with 'ant junit-lowmem' (after building Derby), which uses a 16 MB heap. The test failed when run without the fix, it succeeded with patch 1a applied. Patch ready for review.
          Hide
          Kristian Waagan added a comment -

          Committed patches 1a and 2a to trunk with revision 1160593.
          Backported to 10.8 with revision 1160595.

          I don't expect more work on this issue.

          Show
          Kristian Waagan added a comment - Committed patches 1a and 2a to trunk with revision 1160593. Backported to 10.8 with revision 1160595. I don't expect more work on this issue.
          Hide
          Kathey Marsden added a comment -

          The test testRepeadedDatabaseCreationWithAutoStats has a lot of verbosity with -Dderby.test.trace=true

          testRepeatedDatabaseCreationWithAutoStats – 0
          – 1
          – 2
          – 3
          – 4
          – 5
          – 6
          – 7
          – 8
          – 9
          – 10
          – 11
          – 12
          – 13

          Should this traceit call be prntln so it just prnts with -Dderby.tests.debug=true ?
          while (iter < 50) {

          traceit("-- " + iter++);

          Show
          Kathey Marsden added a comment - The test testRepeadedDatabaseCreationWithAutoStats has a lot of verbosity with -Dderby.test.trace=true testRepeatedDatabaseCreationWithAutoStats – 0 – 1 – 2 – 3 – 4 – 5 – 6 – 7 – 8 – 9 – 10 – 11 – 12 – 13 Should this traceit call be prntln so it just prnts with -Dderby.tests.debug=true ? while (iter < 50) { traceit("-- " + iter++);
          Hide
          Kristian Waagan added a comment -

          Yes. I did so with revision 1173647 (trunk only so far).

          I think it may be wise to also reduce the number of iterations, as the test takes a while to complete.

          Show
          Kristian Waagan added a comment - Yes. I did so with revision 1173647 (trunk only so far). I think it may be wise to also reduce the number of iterations, as the test takes a while to complete.
          Hide
          Myrna van Lunteren added a comment -

          Did you plan to backport this latest fix to the 10.8 branch also? If not I'll do it...

          Show
          Myrna van Lunteren added a comment - Did you plan to backport this latest fix to the 10.8 branch also? If not I'll do it...
          Hide
          Kristian Waagan added a comment -

          Thanks for noticing, Myrna.
          Backported to 10.8 with revision 1174576.

          Show
          Kristian Waagan added a comment - Thanks for noticing, Myrna. Backported to 10.8 with revision 1174576.
          Hide
          Knut Anders Hatlen added a comment -

          [bulk update] Close all resolved issues that haven't been updated for more than one year.

          Show
          Knut Anders Hatlen added a comment - [bulk update] Close all resolved issues that haven't been updated for more than one year.

            People

            • Assignee:
              Kristian Waagan
              Reporter:
              Aja Walker
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development