Qpid
  1. Qpid
  2. QPID-5012

[Java Broker] update optional BDB store to use version 5.0.84 of BDB JE

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.25
    • Component/s: Java Broker
    • Labels:
      None

      Description

      A newer release of BDB JE is available, 5.0.84. We should update the dependency for the optional BDB store to use this. The release contains a number of defect fixes that appear to have the potential to impact users of the Java Broker.

      http://download.oracle.com/otndocs/products/berkeleydb/html/je/je-5.0.84_changelog.html

        Issue Links

          Activity

          Hide
          Keith Wall added a comment -

          Spinning thread was reported to Oracle under this BDB JE forum posting: https://forums.oracle.com/thread/2563876. This posting says this problem will be resolved by the next released (release date tba).

          Show
          Keith Wall added a comment - Spinning thread was reported to Oracle under this BDB JE forum posting: https://forums.oracle.com/thread/2563876 . This posting says this problem will be resolved by the next released (release date tba).
          Hide
          ASF subversion and git services added a comment -

          Commit 1507995 from Robbie Gemmell in branch 'qpid/trunk'
          [ https://svn.apache.org/r1507995 ]

          QPID-5012: use the constant to make the hard-dependency on 5.0.84+ clear at compile-time

          Show
          ASF subversion and git services added a comment - Commit 1507995 from Robbie Gemmell in branch 'qpid/trunk' [ https://svn.apache.org/r1507995 ] QPID-5012 : use the constant to make the hard-dependency on 5.0.84+ clear at compile-time
          Hide
          Robbie Gemmell added a comment -

          I wasn't sure about the use of a literal to avoid the compile dependency on 5.0.84 as it wasn't clear it would actually help, i.e. it depends on whether the names are validated or not. Checking the javadoc suggested they were and upon trying it they are indeed:

          Caused by: java.lang.IllegalArgumentException: je.stats.collectaaa is not a valid BDBJE environment parameter
          

          As a result we should just use the constant and force the compile time dependency since users would have no way of using older versions.

          Show
          Robbie Gemmell added a comment - I wasn't sure about the use of a literal to avoid the compile dependency on 5.0.84 as it wasn't clear it would actually help, i.e. it depends on whether the names are validated or not. Checking the javadoc suggested they were and upon trying it they are indeed: Caused by: java.lang.IllegalArgumentException: je.stats.collectaaa is not a valid BDBJE environment parameter As a result we should just use the constant and force the compile time dependency since users would have no way of using older versions.
          Hide
          ASF subversion and git services added a comment -

          Commit 1506773 from Keith Wall in branch 'qpid/trunk'
          [ https://svn.apache.org/r1506773 ]

          QPID-5012: [Java Broker] Turn off BDB JE stats generation feature (on by default from 5.0.84).

          Show
          ASF subversion and git services added a comment - Commit 1506773 from Keith Wall in branch 'qpid/trunk' [ https://svn.apache.org/r1506773 ] QPID-5012 : [Java Broker] Turn off BDB JE stats generation feature (on by default from 5.0.84).
          Hide
          Keith Wall added a comment -

          I can see that test BDBMessageStoreConfigurationTest is failing to close the BDB Environment. This has the affect of leaving BDB's StatCleaner threads running. These get confused when their associated work directory is removed (they spin repeatedly logging an ISE). I also notice that yesterday's jenkins build 982 was huge as a result of this problem.

          I'll commit a change to trunk to turn of the new stats collection feature (this is the default we'd want).

          Separately, we need to fix BDBMessageStoreConfigurationTest so that the Environment is always closed, and a defect needs to be raised on the BDB JE forums.

          Show
          Keith Wall added a comment - I can see that test BDBMessageStoreConfigurationTest is failing to close the BDB Environment. This has the affect of leaving BDB's StatCleaner threads running. These get confused when their associated work directory is removed (they spin repeatedly logging an ISE). I also notice that yesterday's jenkins build 982 was huge as a result of this problem. I'll commit a change to trunk to turn of the new stats collection feature (this is the default we'd want). Separately, we need to fix BDBMessageStoreConfigurationTest so that the Environment is always closed, and a defect needs to be raised on the BDB JE forums.
          Hide
          Keith Wall added a comment - - edited

          There is a minor issue. It seems Oracle have chosen to enable the new stats capture feature by default, and this is causing the system test to emit exceptions into the logs during test runs. I guess the je thread is unhappy with the fact that the store directories are being deleted (by QpidBrokerTestCase). This is not causing test failures.

          I think we should disable stats collection by default. Users can re-enable if they require this feature.

          [junit] StatCapture 2013-07-23 14:34:35,421 ERROR [server.store.berkeleydb.AbstractBDBMessageStore] Asynchronous exception thrown by BDB thread 'StatCapture'
              [junit] java.lang.IllegalStateException: Error occured accessing statistic log file/tmp/testCreateQueueAMQQueue/je.stat.csv
              [junit] 	at com.sleepycat.utilint.StatLogger.write(StatLogger.java:250)
              [junit] 	at com.sleepycat.utilint.StatLogger.log(StatLogger.java:147)
              [junit] 	at com.sleepycat.je.statcap.StatCapture.outputStats(StatCapture.java:196)
              [junit] 	at com.sleepycat.je.statcap.StatCapture.onWakeup(StatCapture.java:142)
              [junit] 	at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:148)
          
          Show
          Keith Wall added a comment - - edited There is a minor issue. It seems Oracle have chosen to enable the new stats capture feature by default, and this is causing the system test to emit exceptions into the logs during test runs. I guess the je thread is unhappy with the fact that the store directories are being deleted (by QpidBrokerTestCase). This is not causing test failures. I think we should disable stats collection by default. Users can re-enable if they require this feature. [junit] StatCapture 2013-07-23 14:34:35,421 ERROR [server.store.berkeleydb.AbstractBDBMessageStore] Asynchronous exception thrown by BDB thread 'StatCapture' [junit] java.lang.IllegalStateException: Error occured accessing statistic log file/tmp/testCreateQueueAMQQueue/je.stat.csv [junit] at com.sleepycat.utilint.StatLogger.write(StatLogger.java:250) [junit] at com.sleepycat.utilint.StatLogger.log(StatLogger.java:147) [junit] at com.sleepycat.je.statcap.StatCapture.outputStats(StatCapture.java:196) [junit] at com.sleepycat.je.statcap.StatCapture.onWakeup(StatCapture.java:142) [junit] at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:148)
          Hide
          Robbie Gemmell added a comment - - edited

          Change looks good to me. I think we should request this change for inclusion in 0.24.

          When considering this for 0.24, it is important to note this would only affect a developer choosing to compile the optional bdbstore code. We don't ship the jar, users of the java broker binary release have to provide it themselves and so are free to use any compatible version they like. This applies to the maven artifacts as well, since the user has to explicitly add the BDB JE dependency to their own pom and so are again free to utilise whichever compatible version they like.

          EDIT: agreed that we should not request for 0.24 after disussing below items.

          Show
          Robbie Gemmell added a comment - - edited Change looks good to me. I think we should request this change for inclusion in 0.24. When considering this for 0.24, it is important to note this would only affect a developer choosing to compile the optional bdbstore code. We don't ship the jar, users of the java broker binary release have to provide it themselves and so are free to use any compatible version they like. This applies to the maven artifacts as well, since the user has to explicitly add the BDB JE dependency to their own pom and so are again free to utilise whichever compatible version they like. EDIT: agreed that we should not request for 0.24 after disussing below items.
          Hide
          ASF subversion and git services added a comment -

          Commit 1506016 from Keith Wall in branch 'qpid/trunk'
          [ https://svn.apache.org/r1506016 ]

          QPID-5012: update optional BDB store to use version 5.0.84 of BDB JE

          Show
          ASF subversion and git services added a comment - Commit 1506016 from Keith Wall in branch 'qpid/trunk' [ https://svn.apache.org/r1506016 ] QPID-5012 : update optional BDB store to use version 5.0.84 of BDB JE

            People

            • Assignee:
              Keith Wall
              Reporter:
              Keith Wall
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development