Derby
  1. Derby
  2. DERBY-5663

Getting NPE when trying to set derby.language.logStatementText property to true inside a junit suite.

    Details

    • Urgency:
      Normal
    • Issue & fix info:
      Patch Available, Repro attached

      Description

      Derby has a large data suite which runs LobLimitsTest with small data size, large data size and with embedded and network server configurations. The large data suite is run as follows
      time java -Dderby.tests.trace=true -Dderby.infolog.append=true junit.textui.TestRunner org.apache.derbyTesting.functionTests.tests.largedata._Suite > runall.out 2>&1

      I made a simple change to the suite to log statement text as shown in the attached patch(DERBY5663_patch1.txt). This causes the large data suite to run into NPE (NPE can be seen in runall.out) as shown below. Not sure what I am doing wrong while trying to set the property, which results in NPE.
      .
      (emb)largedata.Derby5624Test.testDERBY_5624 used 411473 ms .
      (emb)largedata.LobLimitsTest.test_01_Blob used 1555 ms .
      (emb)largedata.LobLimitsTest.test_02_BlobNegative used 42 ms .
      (emb)largedata.LobLimitsTest.test_03_Clob1 used 1436 ms .
      (emb)largedata.LobLimitsTest.test_04_Clob2 used 1707 ms .
      (emb)largedata.LobLimitsTest.test_05_ClobNegative used 967 ms E.
      (emb)largedata.LobLimitsTest.test_01_Blob used 2929139 ms .
      (emb)largedata.LobLimitsTest.test_02_BlobNegative used 154 ms .
      (emb)largedata.LobLimitsTest.test_03_Clob1 used 2854121 ms .
      (emb)largedata.LobLimitsTest.test_04_Clob2 used 656137 ms .
      (emb)largedata.LobLimitsTest.test_05_ClobNegative used 331288 ms EF
      Time: 7,589.168
      There were 2 errors:
      1) LobLimitsTestjava.lang.NullPointerException
      at org.apache.derbyTesting.junit.SystemPropertyTestSetup.setProperties(SystemPropertyTestSetup.java:116)
      at org.apache.derbyTesting.junit.SystemPropertyTestSetup.setUp(SystemPropertyTestSetup.java:87)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:18)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      2) LobLimitsTestjava.sql.SQLNonTransientConnectionException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: org.apache.derby.jdbc.EmbeddedDriver is not registered with the JDBC driver manager
      at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
      at org.apache.derby.client.am.SqlException.getSQLException(SqlException.java:364)
      at org.apache.derby.jdbc.ClientDriver.connect(ClientDriver.java:166)
      at java.sql.DriverManager.getConnection(DriverManager.java:322)
      at java.sql.DriverManager.getConnection(DriverManager.java:297)
      at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:100)
      at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:67)
      at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43)
      at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:1633)
      at org.apache.derbyTesting.junit.BaseJDBCTestSetup.getConnection(BaseJDBCTestSetup.java:72)
      at org.apache.derbyTesting.junit.CleanDatabaseTestSetup.setUp(CleanDatabaseTestSetup.java:104)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:18)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: org.apache.derby.jdbc.EmbeddedDriver is not registered with the JDBC driver manager
      at org.apache.derby.client.am.Connection.completeSqlca(Connection.java:2125)
      at org.apache.derby.client.net.NetConnectionReply.parseRdbAccessFailed(NetConnectionReply.java:538)
      at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(NetConnectionReply.java:431)
      at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(NetConnectionReply.java:294)
      at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(NetConnectionReply.java:121)
      at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(NetConnection.java:826)
      at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(NetConnection.java:762)
      at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(NetConnection.java:591)
      at org.apache.derby.client.net.NetConnection.flowConnect(NetConnection.java:406)
      at org.apache.derby.client.net.NetConnection.<init>(NetConnection.java:220)
      at org.apache.derby.client.net.NetConnection40.<init>(NetConnection40.java:74)
      at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl40.newNetConnection(ClientJDBCObjectFactoryImpl40.java:269)
      at org.apache.derby.jdbc.ClientDriver.connect(ClientDriver.java:157)
      ... 43 more
      There was 1 failure:
      1) LobLimitsTestjunit.framework.ComparisonFailure: Engine shutdown expected:<XJ015> but was:<08001>
      at org.apache.derbyTesting.junit.BaseJDBCTestCase.assertSQLState(BaseJDBCTestCase.java:790)
      at org.apache.derbyTesting.junit.TestConfiguration.shutdownEngine(TestConfiguration.java:1751)
      at org.apache.derbyTesting.junit.SystemPropertyTestSetup.tearDown(SystemPropertyTestSetup.java:108)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:20)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      Caused by: java.sql.SQLException: No suitable driver
      at java.sql.DriverManager.getConnection(DriverManager.java:330)
      at java.sql.DriverManager.getConnection(DriverManager.java:297)
      at org.apache.derbyTesting.junit.DriverManagerConnector.getConnectionByAttributes(DriverManagerConnector.java:163)
      at org.apache.derbyTesting.junit.DriverManagerConnector.shutEngine(DriverManagerConnector.java:140)
      at org.apache.derbyTesting.junit.TestConfiguration.shutdownEngine(TestConfiguration.java:1748)
      ... 31 more

      FAILURES!!!
      Tests run: 11, Failures: 1, Errors: 2

      1. DERBY5663_patch1.txt
        2 kB
        Mamta A. Satoor
      2. DERBY5663_patch2.txt
        2 kB
        Mamta A. Satoor

        Issue Links

          Activity

          Hide
          Mamta A. Satoor added a comment -

          The NPE seems to happen after the last fixture of the previous suite is finished and we are starting the new suite.

          Show
          Mamta A. Satoor added a comment - The NPE seems to happen after the last fixture of the previous suite is finished and we are starting the new suite.
          Hide
          Kathey Marsden added a comment -

          Just glancing at the code, I wonder if you have gotten into a situation where the SystemPropertyTest decorator instance is somehow getting run twice. It may be related to the order of decorators. I am curious though what happens if you comment out this line:
          // newValues = null;

          In the tearDown() method of SystemPropertyTestSetup.

          Show
          Kathey Marsden added a comment - Just glancing at the code, I wonder if you have gotten into a situation where the SystemPropertyTest decorator instance is somehow getting run twice. It may be related to the order of decorators. I am curious though what happens if you comment out this line: // newValues = null; In the tearDown() method of SystemPropertyTestSetup.
          Hide
          Mamta A. Satoor added a comment -

          Kathey, I made the code change suggested by you but it still results into NPE.

          I looked at the line of the code(SystemPropertyTestSetup.tearDown()) where NPE is coming and it is as follows
          if (oldValues.getProperty(key) == null)

          oldValues is null which results into NPE. This happens because I think we have set oldValues = null; during the previous instance of the decorator.

          I am going to change oldValues = null; to oldValues = new Properties(); and see how that goes(I will leave the comment around newValues = null; )

          Show
          Mamta A. Satoor added a comment - Kathey, I made the code change suggested by you but it still results into NPE. I looked at the line of the code(SystemPropertyTestSetup.tearDown()) where NPE is coming and it is as follows if (oldValues.getProperty(key) == null) oldValues is null which results into NPE. This happens because I think we have set oldValues = null; during the previous instance of the decorator. I am going to change oldValues = null; to oldValues = new Properties(); and see how that goes(I will leave the comment around newValues = null; )
          Hide
          Mamta A. Satoor added a comment -

          I ran large data suite with following changes in SystemPropertyTestSetup.tearDown()
          — java/testing/org/apache/derbyTesting/junit/SystemPropertyTestSetup.java (revision 1305533)
          +++ java/testing/org/apache/derbyTesting/junit/SystemPropertyTestSetup.java (working copy)
          @@ -106,8 +106,8 @@
          // shutdown engine to restore any static properties
          if (staticProperties)
          TestConfiguration.getCurrent().shutdownEngine();

          • newValues = null;
          • oldValues = null;
            +// newValues = null;
            + oldValues = new Properties();
            }

          I don't get NPE anymore but I still get following failure/error while running large data suite with SystemPropertyTestSetup
          1) LobLimitsTestjava.sql.SQLNonTransientConnectionException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: org.apache.derby.jdbc.EmbeddedDriver is not registered with the JDBC driver manager
          at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71)
          at org.apache.derby.client.am.SqlException.getSQLException(SqlException.java:364)
          at org.apache.derby.jdbc.ClientDriver.connect(ClientDriver.java:166)
          at java.sql.DriverManager.getConnection(DriverManager.java:322)
          at java.sql.DriverManager.getConnection(DriverManager.java:297)
          at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:100)
          at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:67)
          at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43)
          at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:1633)
          at org.apache.derbyTesting.junit.BaseJDBCTestSetup.getConnection(BaseJDBCTestSetup.java:72)
          at org.apache.derbyTesting.junit.CleanDatabaseTestSetup.setUp(CleanDatabaseTestSetup.java:104)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:18)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: org.apache.derby.jdbc.EmbeddedDriver is not registered with the JDBC driver manager
          at org.apache.derby.client.am.Connection.completeSqlca(Connection.java:2125)
          at org.apache.derby.client.net.NetConnectionReply.parseRdbAccessFailed(NetConnectionReply.java:538)
          at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(NetConnectionReply.java:431)
          at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(NetConnectionReply.java:294)
          at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(NetConnectionReply.java:121)
          at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(NetConnection.java:826)
          at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(NetConnection.java:762)
          at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(NetConnection.java:591)
          at org.apache.derby.client.net.NetConnection.flowConnect(NetConnection.java:406)
          at org.apache.derby.client.net.NetConnection.<init>(NetConnection.java:220)
          at org.apache.derby.client.net.NetConnection40.<init>(NetConnection40.java:74)
          at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl40.newNetConnection(ClientJDBCObjectFactoryImpl40.java:269)
          at org.apache.derby.jdbc.ClientDriver.connect(ClientDriver.java:157)
          ... 36 more
          There was 1 failure:
          1) LobLimitsTestjunit.framework.ComparisonFailure: Engine shutdown expected:<XJ015> but was:<08001>
          at org.apache.derbyTesting.junit.BaseJDBCTestCase.assertSQLState(BaseJDBCTestCase.java:790)
          at org.apache.derbyTesting.junit.TestConfiguration.shutdownEngine(TestConfiguration.java:1751)
          at org.apache.derbyTesting.junit.SystemPropertyTestSetup.tearDown(SystemPropertyTestSetup.java:108)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:20)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          Caused by: java.sql.SQLException: No suitable driver
          at java.sql.DriverManager.getConnection(DriverManager.java:330)
          at java.sql.DriverManager.getConnection(DriverManager.java:297)
          at org.apache.derbyTesting.junit.DriverManagerConnector.getConnectionByAttributes(DriverManagerConnector.java:163)
          at org.apache.derbyTesting.junit.DriverManagerConnector.shutEngine(DriverManagerConnector.java:140)
          at org.apache.derbyTesting.junit.TestConfiguration.shutdownEngine(TestConfiguration.java:1748)
          ... 24 more

          FAILURES!!!

          Show
          Mamta A. Satoor added a comment - I ran large data suite with following changes in SystemPropertyTestSetup.tearDown() — java/testing/org/apache/derbyTesting/junit/SystemPropertyTestSetup.java (revision 1305533) +++ java/testing/org/apache/derbyTesting/junit/SystemPropertyTestSetup.java (working copy) @@ -106,8 +106,8 @@ // shutdown engine to restore any static properties if (staticProperties) TestConfiguration.getCurrent().shutdownEngine(); newValues = null; oldValues = null; +// newValues = null; + oldValues = new Properties(); } I don't get NPE anymore but I still get following failure/error while running large data suite with SystemPropertyTestSetup 1) LobLimitsTestjava.sql.SQLNonTransientConnectionException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: org.apache.derby.jdbc.EmbeddedDriver is not registered with the JDBC driver manager at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:71) at org.apache.derby.client.am.SqlException.getSQLException(SqlException.java:364) at org.apache.derby.jdbc.ClientDriver.connect(ClientDriver.java:166) at java.sql.DriverManager.getConnection(DriverManager.java:322) at java.sql.DriverManager.getConnection(DriverManager.java:297) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:100) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:67) at org.apache.derbyTesting.junit.DriverManagerConnector.openConnection(DriverManagerConnector.java:43) at org.apache.derbyTesting.junit.TestConfiguration.openDefaultConnection(TestConfiguration.java:1633) at org.apache.derbyTesting.junit.BaseJDBCTestSetup.getConnection(BaseJDBCTestSetup.java:72) at org.apache.derbyTesting.junit.CleanDatabaseTestSetup.setUp(CleanDatabaseTestSetup.java:104) at junit.extensions.TestSetup$1.protect(TestSetup.java:18) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 08006, SQLERRMC: org.apache.derby.jdbc.EmbeddedDriver is not registered with the JDBC driver manager at org.apache.derby.client.am.Connection.completeSqlca(Connection.java:2125) at org.apache.derby.client.net.NetConnectionReply.parseRdbAccessFailed(NetConnectionReply.java:538) at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(NetConnectionReply.java:431) at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(NetConnectionReply.java:294) at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(NetConnectionReply.java:121) at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(NetConnection.java:826) at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(NetConnection.java:762) at org.apache.derby.client.net.NetConnection.flowUSRIDPWDconnect(NetConnection.java:591) at org.apache.derby.client.net.NetConnection.flowConnect(NetConnection.java:406) at org.apache.derby.client.net.NetConnection.<init>(NetConnection.java:220) at org.apache.derby.client.net.NetConnection40.<init>(NetConnection40.java:74) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl40.newNetConnection(ClientJDBCObjectFactoryImpl40.java:269) at org.apache.derby.jdbc.ClientDriver.connect(ClientDriver.java:157) ... 36 more There was 1 failure: 1) LobLimitsTestjunit.framework.ComparisonFailure: Engine shutdown expected:<XJ015> but was:<08001> at org.apache.derbyTesting.junit.BaseJDBCTestCase.assertSQLState(BaseJDBCTestCase.java:790) at org.apache.derbyTesting.junit.TestConfiguration.shutdownEngine(TestConfiguration.java:1751) at org.apache.derbyTesting.junit.SystemPropertyTestSetup.tearDown(SystemPropertyTestSetup.java:108) at junit.extensions.TestSetup$1.protect(TestSetup.java:20) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) Caused by: java.sql.SQLException: No suitable driver at java.sql.DriverManager.getConnection(DriverManager.java:330) at java.sql.DriverManager.getConnection(DriverManager.java:297) at org.apache.derbyTesting.junit.DriverManagerConnector.getConnectionByAttributes(DriverManagerConnector.java:163) at org.apache.derbyTesting.junit.DriverManagerConnector.shutEngine(DriverManagerConnector.java:140) at org.apache.derbyTesting.junit.TestConfiguration.shutdownEngine(TestConfiguration.java:1748) ... 24 more FAILURES!!!
          Hide
          Mamta A. Satoor added a comment -

          I found that I was using SystemPropertyTestSetup decorator such that it would shutdown the engine before setting the passed properties. I do not need the engine to be shutdown and hence changed the invocation of SystemPropertyTestSetup from
          suite = new SystemPropertyTestSetup(suite,sysprops,true);
          to
          suite = new SystemPropertyTestSetup(suite,sysprops);

          After this change, I do not get following failure
          LobLimitsTestjunit.framework.ComparisonFailure: Engine shutdown expected:<XJ015> but was:<08001>

          Show
          Mamta A. Satoor added a comment - I found that I was using SystemPropertyTestSetup decorator such that it would shutdown the engine before setting the passed properties. I do not need the engine to be shutdown and hence changed the invocation of SystemPropertyTestSetup from suite = new SystemPropertyTestSetup(suite,sysprops,true); to suite = new SystemPropertyTestSetup(suite,sysprops); After this change, I do not get following failure LobLimitsTestjunit.framework.ComparisonFailure: Engine shutdown expected:<XJ015> but was:<08001>
          Hide
          Mamta A. Satoor added a comment -

          Attaching patch DERBY5663_patch2.txt for this issue. The patch touches following 2 files.
          $ svn stat -q
          M java\testing\org\apache\derbyTesting\functionTests\tests\largedata\LobLimitsTest.java
          M java\testing\org\apache\derbyTesting\junit\SystemPropertyTestSetup.java

          Changes in LobLimitsTest.java now adds properties related to lock timeouts. In case the test ran into lock timeouts as noted by DERBY-5368, we should have relevant information in derby.log to debug the problem.

          Changes in SystemPropertyTestSetup.java.
          Two of the variables defined in this class are
          protected Properties newValues;
          private Properties oldValues;
          newValues are the new properties requested by the junit suite through the SystemPropertyTestSetup decorator. These values become effective in SystemPropertyTestSetup.setUp() method.
          oldValues are the original properties which we want to revert back to at the time of teardown of this decorator.

          In trunk, both oldValues and newValues get set to null at the end of the tearDown which causes two problems
          1)npe in following piece of code in SystemPropertyTestSetup.tearDown()
          if (oldValues.getProperty(key) == null)
          Solution to npe is that rather than nulling oldValues, we should initialize it to empty properties object(similar to what happens in the decorator constructor). To fix npe, in SystemPropertyTestSetup.tearDown(), I have replaced
          oldValues = null;
          with
          oldValues = new Properties();
          2)new properties not getting reused when the same decorator instance is used for another suite. It is possible to use the same instance of SystemPropertyTestSetup for multiple runs of a suite.
          LobLimitsTest in the patch uses SystemPropertyTestSetup decorator to set locking related properties as shown below(attached patch shows these changes to LobLimitsTest)
          static Test baseSuite(final int biggestSize, final int bigSize) {
          //Run the suite with following properties in case we run into lock
          // time out issues. It will help debug the problem if timeouts occur.
          Properties sysprops = new Properties();
          sysprops.setProperty("derby.locks.deadlockTrace", "true");
          sysprops.setProperty("derby.locks.monitor", "true");
          // Some of the test cases depend on certain other test cases to run
          // first, so force the test cases to run in lexicographical order.
          Test suite = new CleanDatabaseTestSetup(
          TestConfiguration.orderedSuite(LobLimitsTest.class)) {
          protected void decorateSQL(Statement s)
          throws SQLException

          { setupTables(s, biggestSize, bigSize); }

          };
          suite = new SystemPropertyTestSetup(suite,sysprops);//,true);

          return new SupportFilesSetup(suite);
          }

          Next, LobLimitsTest gets called by tests.largedata._Suite 4 times, one time each for different large object sizes and for embedded and network server runs. Nulling of newValues in SystemPropertyTestSetup.tearDown causes the subsequent use of SystemPropertyTestSetup instance to loose the properties requested by LobLimitsTest. In order to fix this, we should not null the newValues properties in SystemPropertyTestSetup.tearDown()

          I will run the entire junit suite one time to make sure nothing has broken as a result of these changes. Thanks

          Show
          Mamta A. Satoor added a comment - Attaching patch DERBY5663_patch2.txt for this issue. The patch touches following 2 files. $ svn stat -q M java\testing\org\apache\derbyTesting\functionTests\tests\largedata\LobLimitsTest.java M java\testing\org\apache\derbyTesting\junit\SystemPropertyTestSetup.java Changes in LobLimitsTest.java now adds properties related to lock timeouts. In case the test ran into lock timeouts as noted by DERBY-5368 , we should have relevant information in derby.log to debug the problem. Changes in SystemPropertyTestSetup.java. Two of the variables defined in this class are protected Properties newValues; private Properties oldValues; newValues are the new properties requested by the junit suite through the SystemPropertyTestSetup decorator. These values become effective in SystemPropertyTestSetup.setUp() method. oldValues are the original properties which we want to revert back to at the time of teardown of this decorator. In trunk, both oldValues and newValues get set to null at the end of the tearDown which causes two problems 1)npe in following piece of code in SystemPropertyTestSetup.tearDown() if (oldValues.getProperty(key) == null) Solution to npe is that rather than nulling oldValues, we should initialize it to empty properties object(similar to what happens in the decorator constructor). To fix npe, in SystemPropertyTestSetup.tearDown(), I have replaced oldValues = null; with oldValues = new Properties(); 2)new properties not getting reused when the same decorator instance is used for another suite. It is possible to use the same instance of SystemPropertyTestSetup for multiple runs of a suite. LobLimitsTest in the patch uses SystemPropertyTestSetup decorator to set locking related properties as shown below(attached patch shows these changes to LobLimitsTest) static Test baseSuite(final int biggestSize, final int bigSize) { //Run the suite with following properties in case we run into lock // time out issues. It will help debug the problem if timeouts occur. Properties sysprops = new Properties(); sysprops.setProperty("derby.locks.deadlockTrace", "true"); sysprops.setProperty("derby.locks.monitor", "true"); // Some of the test cases depend on certain other test cases to run // first, so force the test cases to run in lexicographical order. Test suite = new CleanDatabaseTestSetup( TestConfiguration.orderedSuite(LobLimitsTest.class)) { protected void decorateSQL(Statement s) throws SQLException { setupTables(s, biggestSize, bigSize); } }; suite = new SystemPropertyTestSetup(suite,sysprops);//,true); return new SupportFilesSetup(suite); } Next, LobLimitsTest gets called by tests.largedata._Suite 4 times, one time each for different large object sizes and for embedded and network server runs. Nulling of newValues in SystemPropertyTestSetup.tearDown causes the subsequent use of SystemPropertyTestSetup instance to loose the properties requested by LobLimitsTest. In order to fix this, we should not null the newValues properties in SystemPropertyTestSetup.tearDown() I will run the entire junit suite one time to make sure nothing has broken as a result of these changes. Thanks
          Hide
          Knut Anders Hatlen added a comment - - edited

          I'm not so comfortable with the suggested changes in SystemPropertyTestSetup, as they look more like a workaround for a bug than a fix for the root cause. The changes will prevent freeing the contents of newValues for all tests that use SystemPropertyTestSetup, which will probably increase the memory requirements for suites.All.

          Did you get any closer to finding out why it would end up running setUp() after tearDown()?

          I didn't quite get the part about largedata._Suite calling LobLimitTest four times, and that tearDown() in one test would null out the references for the next test. There should be one instance of SystemPropertyTestSetup per instance of LobLimitTest, and one instance of SystemPropertyTestSetup should not null out the fields in another instance, or am I misunderstanding?

          [comment edited: added missing not in the last sentence]

          Show
          Knut Anders Hatlen added a comment - - edited I'm not so comfortable with the suggested changes in SystemPropertyTestSetup, as they look more like a workaround for a bug than a fix for the root cause. The changes will prevent freeing the contents of newValues for all tests that use SystemPropertyTestSetup, which will probably increase the memory requirements for suites.All. Did you get any closer to finding out why it would end up running setUp() after tearDown()? I didn't quite get the part about largedata._Suite calling LobLimitTest four times, and that tearDown() in one test would null out the references for the next test. There should be one instance of SystemPropertyTestSetup per instance of LobLimitTest, and one instance of SystemPropertyTestSetup should not null out the fields in another instance, or am I misunderstanding? [comment edited: added missing not in the last sentence]
          Hide
          Mamta A. Satoor added a comment -

          Hi Knut, first of all, thanks for looking at this jira.

          I am debugging further to get the exact details as to why setUp() is getting called after tearDown() but I believe it is because the same instance of SystemPropertyTestSetup is getting twice.

          Show
          Mamta A. Satoor added a comment - Hi Knut, first of all, thanks for looking at this jira. I am debugging further to get the exact details as to why setUp() is getting called after tearDown() but I believe it is because the same instance of SystemPropertyTestSetup is getting twice.
          Hide
          Mamta A. Satoor added a comment -

          Let me start first by just going over the large data suite at the top level.

          tests.largedata._Suite has following
          public static Test suite()

          { TestSuite suite = new TestSuite("largedata suite"); // DERBY-5624, currently this runs out of file descriptors on unix // systems with 1024 limit per user. Setting to run only on windows // until solution for unix is found. if (isWindowsPlatform()) suite.addTest(Derby5624Test.suite()); suite.addTest(LobLimitsLiteTest.suite()); suite.addTest(LobLimitsTest.suite()); suite.addTest(LobLimitsClientTest.suite()); return suite; }

          For the sake of this jira, we can ignore Derby5624Test.suite since this suite has test fixture of it's own.

          The next three suites are
          1)LobLimitsLiteTest
          2)LobLimitsTest and
          3)LobLimitsClientTest

          All of the three suites above end up calling tests.LobLimitsTest suite.

          LobLimitsLiteTest specifically calls LobLimitsTest suite twice(once for embedded and once for network server) as shown below
          The suite() method in LobLimitsLiteTest is as follows
          public static Test suite()

          { Test test = LobLimitsTest.baseSuite(_1MB, _100K); TestSuite suite = new TestSuite("LobLimitsLiteTest"); suite.addTest(test); suite.addTest(TestConfiguration.clientServerDecorator(test)); return suite; }

          As we can see, we create only one instance of LobLimitesTest suite and run that once in embedded and next in client server mode. Since we are using the same instance of LobLimitesTest suite twice, in turn we are using the same SystemPropertyTestSetup instance twice.

          As for the next 2 suites in tests.largedata._Suite(namely LobLimitsTest and LobLimitsClientTest), they create their own LobLimitsTest suite and hence there is no reuse of the same instance of SystemPropertyTestSetup.

          I verified this by putting println in SystemPropertyTestSetup's constructor and found that the println gets printed only 3 times(rather than 4 times which is how many times the LobLimitesTest suite is invoked).
          System.out.println("in constructor " + hashCode());
          I also put a println in SystemPropertyTestSetup.setUP() and see that it got called 4 times. But 2 of those 4 times printed the same hashCode() indicating we are dealing with the same instance of SystemPropertyTestSetup.
          System.out.println("in setUp " + hashCode());

          I modified LobLimitsLiteTest.suite() just for testing purposes to create a new suite LobLimitsLiteTest for client server run and after that change, I see the SystemPropertyTestSetup's constructor getting called 4 times. With this temporary change in my codeline with each lob limits test using it's own instance, it doesn't matter if we null out oldValue and newValues in SystemPropertyTestSetup.tearDown() method since there is no resue of the same instance of SystemPropertyTestSetup
          public static Test suite()

          { Test test = LobLimitsTest.baseSuite(_1MB, _100K); TestSuite suite = new TestSuite("LobLimitsLiteTest"); suite.addTest(test); Test test1 = LobLimitsTest.baseSuite(_1MB, _100K); suite.addTest(TestConfiguration.clientServerDecorator(test1)); return suite; }
          Show
          Mamta A. Satoor added a comment - Let me start first by just going over the large data suite at the top level. tests.largedata._Suite has following public static Test suite() { TestSuite suite = new TestSuite("largedata suite"); // DERBY-5624, currently this runs out of file descriptors on unix // systems with 1024 limit per user. Setting to run only on windows // until solution for unix is found. if (isWindowsPlatform()) suite.addTest(Derby5624Test.suite()); suite.addTest(LobLimitsLiteTest.suite()); suite.addTest(LobLimitsTest.suite()); suite.addTest(LobLimitsClientTest.suite()); return suite; } For the sake of this jira, we can ignore Derby5624Test.suite since this suite has test fixture of it's own. The next three suites are 1)LobLimitsLiteTest 2)LobLimitsTest and 3)LobLimitsClientTest All of the three suites above end up calling tests.LobLimitsTest suite. LobLimitsLiteTest specifically calls LobLimitsTest suite twice(once for embedded and once for network server) as shown below The suite() method in LobLimitsLiteTest is as follows public static Test suite() { Test test = LobLimitsTest.baseSuite(_1MB, _100K); TestSuite suite = new TestSuite("LobLimitsLiteTest"); suite.addTest(test); suite.addTest(TestConfiguration.clientServerDecorator(test)); return suite; } As we can see, we create only one instance of LobLimitesTest suite and run that once in embedded and next in client server mode. Since we are using the same instance of LobLimitesTest suite twice, in turn we are using the same SystemPropertyTestSetup instance twice. As for the next 2 suites in tests.largedata._Suite(namely LobLimitsTest and LobLimitsClientTest), they create their own LobLimitsTest suite and hence there is no reuse of the same instance of SystemPropertyTestSetup. I verified this by putting println in SystemPropertyTestSetup's constructor and found that the println gets printed only 3 times(rather than 4 times which is how many times the LobLimitesTest suite is invoked). System.out.println("in constructor " + hashCode()); I also put a println in SystemPropertyTestSetup.setUP() and see that it got called 4 times. But 2 of those 4 times printed the same hashCode() indicating we are dealing with the same instance of SystemPropertyTestSetup. System.out.println("in setUp " + hashCode()); I modified LobLimitsLiteTest.suite() just for testing purposes to create a new suite LobLimitsLiteTest for client server run and after that change, I see the SystemPropertyTestSetup's constructor getting called 4 times. With this temporary change in my codeline with each lob limits test using it's own instance, it doesn't matter if we null out oldValue and newValues in SystemPropertyTestSetup.tearDown() method since there is no resue of the same instance of SystemPropertyTestSetup public static Test suite() { Test test = LobLimitsTest.baseSuite(_1MB, _100K); TestSuite suite = new TestSuite("LobLimitsLiteTest"); suite.addTest(test); Test test1 = LobLimitsTest.baseSuite(_1MB, _100K); suite.addTest(TestConfiguration.clientServerDecorator(test1)); return suite; }
          Hide
          Kathey Marsden added a comment -

          Thanks Mamta for explaining why the instance is used more than once. I think from a practical perspective it is good to have it work both ways.

          Looking at SystemPropertyTestSetup, I think your change to remove the line setting newValues to null and inititializing odValues to new Properties() will work. This will leave the instance in the correct state for the second iteration as the initialization where newValues has the new values that we want and oldValues is empty and ready to be populated when setProperties() is called by setup().

          Perhaps a more intuitive fix would be to move the initialization of oldValues to setup() rather than the constructor or tearDown() and add a comment there with the bug number, indicating oldValues needs to be initialized for each iteration if the instance is reused.

          Thanks for fixing this up!

          Show
          Kathey Marsden added a comment - Thanks Mamta for explaining why the instance is used more than once. I think from a practical perspective it is good to have it work both ways. Looking at SystemPropertyTestSetup, I think your change to remove the line setting newValues to null and inititializing odValues to new Properties() will work. This will leave the instance in the correct state for the second iteration as the initialization where newValues has the new values that we want and oldValues is empty and ready to be populated when setProperties() is called by setup(). Perhaps a more intuitive fix would be to move the initialization of oldValues to setup() rather than the constructor or tearDown() and add a comment there with the bug number, indicating oldValues needs to be initialized for each iteration if the instance is reused. Thanks for fixing this up!
          Hide
          Knut Anders Hatlen added a comment -

          Thanks for that explanation, Mamta. I agree with Kathey that it would be more intuitive to move the initialization to setUp(), and that would allow us to continue nulling out newValues and oldValues in tearDown() in order to free up memory when the test is done. Note that DatabasePropertyTestSetup uses the same pattern as SystemPropertyTestSetup and would probably need to be changed too, if we think reuse of tests/decorators is OK.

          Show
          Knut Anders Hatlen added a comment - Thanks for that explanation, Mamta. I agree with Kathey that it would be more intuitive to move the initialization to setUp(), and that would allow us to continue nulling out newValues and oldValues in tearDown() in order to free up memory when the test is done. Note that DatabasePropertyTestSetup uses the same pattern as SystemPropertyTestSetup and would probably need to be changed too, if we think reuse of tests/decorators is OK.
          Hide
          Mamta A. Satoor added a comment -

          Kathey and Knut, thanks for looking at the patch.

          1)I will remove nulling out of oldValues and nullValues from tearDown().
          2)I will also remove the intialization of oldValues to new Properties object from the constructor and move it to setUp().

          Knut, we do not want to null newValues in tearDown because that will make us loose the new properties requested by the user when the same instance of SystemPropertyTestSetup gets used again.

          I will run the tests with all these changes.

          Show
          Mamta A. Satoor added a comment - Kathey and Knut, thanks for looking at the patch. 1)I will remove nulling out of oldValues and nullValues from tearDown(). 2)I will also remove the intialization of oldValues to new Properties object from the constructor and move it to setUp(). Knut, we do not want to null newValues in tearDown because that will make us loose the new properties requested by the user when the same instance of SystemPropertyTestSetup gets used again. I will run the tests with all these changes.
          Hide
          Mamta A. Satoor added a comment -

          The changes went in as revision 1309244

          Show
          Mamta A. Satoor added a comment - The changes went in as revision 1309244
          Hide
          Kathey Marsden added a comment -

          Reopen for backport analysis. Temporarily assign to yourself if you backport and then reassign to Mamta before closing.

          Show
          Kathey Marsden added a comment - Reopen for backport analysis. Temporarily assign to yourself if you backport and then reassign to Mamta before closing.
          Hide
          Mamta A. Satoor added a comment -

          I will look at backporting this

          Show
          Mamta A. Satoor added a comment - I will look at backporting this
          Hide
          Mamta A. Satoor added a comment -

          Backported into 10.8.2.3 with revision 1386677

          Show
          Mamta A. Satoor added a comment - Backported into 10.8.2.3 with revision 1386677
          Hide
          Mamta A. Satoor added a comment -

          Backported changes(without the test changes) into 10.7.1.4 with revision 1386854. The test changed(largedata/LobLimitsTest.java) does not exist in 10.7 and earlier releases

          Show
          Mamta A. Satoor added a comment - Backported changes(without the test changes) into 10.7.1.4 with revision 1386854. The test changed(largedata/LobLimitsTest.java) does not exist in 10.7 and earlier releases
          Hide
          Mamta A. Satoor added a comment -

          Backported changes(without the test changes) into 10.6.2.3 with revision 1386967. The test changed(largedata/LobLimitsTest.java) does not exist in 10.7 and earlier releases

          Show
          Mamta A. Satoor added a comment - Backported changes(without the test changes) into 10.6.2.3 with revision 1386967. The test changed(largedata/LobLimitsTest.java) does not exist in 10.7 and earlier releases
          Hide
          Mamta A. Satoor added a comment -

          Backported changes(without the test changes) into 10.5.3.2 with revision 1387211. The test changed(largedata/LobLimitsTest.java) does not exist in 10.7 and earlier releases

          Show
          Mamta A. Satoor added a comment - Backported changes(without the test changes) into 10.5.3.2 with revision 1387211. The test changed(largedata/LobLimitsTest.java) does not exist in 10.7 and earlier releases
          Hide
          Mamta A. Satoor added a comment -

          I am not planning on backporting it any further

          Show
          Mamta A. Satoor added a comment - I am not planning on backporting it any further

            People

            • Assignee:
              Mamta A. Satoor
              Reporter:
              Mamta A. Satoor
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development