Derby
  1. Derby
  2. DERBY-4635

intermittent failure in (...)replicationTests.ReplicationRun_Local_1Indexing.testReplication_Local_1_Indexing

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 10.6.1.0, 10.7.1.1
    • Fix Version/s: None
    • Component/s: Test
    • Urgency:
      Normal
    • Bug behavior facts:
      Regression Test Failure

      Description

      During test cycle for 10.6 RC1 (10.6.1.0 build 938214) on windows (win2008, ibm15 sr10) I saw a failure in org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing.testReplication_Local_1_Indexing.

      This same failure was also reported to the derby-dev list in this thread:
      http://mail-archives.apache.org/mod_mbox/db-derby-dev/201004.mbox/%3c4BBB1323.9090806@sun.com%3e

      Here's my stack trace (I modified the physical paths):

      1) testReplication_Local_1_Indexing(org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing)java.sql.SQLException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'C:\jartest\db_master\wombat', see the next exception for details.::SQLSTATE: XBM0JDirectory C:\jartest\db_master\C:\jartest\db_master\wombat already exists.
      at org.apache.derby.client.am.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source)
      at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source)
      at java.sql.DriverManager.getConnection(DriverManager.java:572)
      at java.sql.DriverManager.getConnection(DriverManager.java:218)
      at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.bootMasterDatabase(ReplicationRun.java:863)
      at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.makeReadyForReplication(ReplicationRun.java:2746)
      at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing.testReplication_Local_1_Indexing(ReplicationRun_Local_1Indexing.java:70)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
      at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.runBare(ReplicationRun.java:207)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'C:\jartest\db_master\wombat', see the next exception for details.::SQLSTATE: XBM0JDirectory C:\jartest\db_master\C:\jartest\db_master\wombat already exists.
      at org.apache.derby.client.am.Connection.completeSqlca(Unknown Source)
      at org.apache.derby.client.net.NetConnectionReply.parseRdbAccessFailed(Unknown Source)
      at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(Unknown Source)
      at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(Unknown Source)
      at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(Unknown Source)
      at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(Unknown Source)
      at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(Unknown Source)
      at org.apache.derby.client.net.NetConnection.flowUSRIDONLconnect(Unknown Source)
      at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source)
      at org.apache.derby.client.net.NetConnection.<init>(Unknown Source)
      at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source)
      ... 33 more

        Issue Links

          Activity

          Hide
          Knut Anders Hatlen added a comment -

          Discussed this issue briefly with Kathey and Lily on IRC. Some thoughts:

          ReplicationRun.makeReadyForReplication() has called initMaster() before it fails in bootMasterDatabase(). initMaster() is supposed to remove all old files in the db_master directory, but it cannot have done so in this case, otherwise we wouldn't have seen this error. Unfortunately, it only prints a message to the console if it cannot delete a file.

          It would probably be better if it just failed immediately when it failed to delete a file, as that would make it easier to pinpoint the cause of the failure. And even better if each test case had cleaned the db_master directory after it has completed instead of before it has started, so that we can tell exactly which test case is not shutting down cleanly.

          Show
          Knut Anders Hatlen added a comment - Discussed this issue briefly with Kathey and Lily on IRC. Some thoughts: ReplicationRun.makeReadyForReplication() has called initMaster() before it fails in bootMasterDatabase(). initMaster() is supposed to remove all old files in the db_master directory, but it cannot have done so in this case, otherwise we wouldn't have seen this error. Unfortunately, it only prints a message to the console if it cannot delete a file. It would probably be better if it just failed immediately when it failed to delete a file, as that would make it easier to pinpoint the cause of the failure. And even better if each test case had cleaned the db_master directory after it has completed instead of before it has started, so that we can tell exactly which test case is not shutting down cleanly.
          Hide
          Kristian Waagan added a comment -

          Maybe the new method PrivilegedFileOpsForTests.persistentRecursiveDelete (or BaseJDBCTestCase.assertDirectoryDeleted) can be used? It will try to delete all files, and report those it couldn't delete.

          Show
          Kristian Waagan added a comment - Maybe the new method PrivilegedFileOpsForTests.persistentRecursiveDelete (or BaseJDBCTestCase.assertDirectoryDeleted) can be used? It will try to delete all files, and report those it couldn't delete.
          Hide
          Knut Anders Hatlen added a comment -

          Yes, that sounds like a good idea. In fact, it looks like there are many methods in the replications tests' Utils class that could be replaced with methods in PrivilegedFileOpsForTests.

          Show
          Knut Anders Hatlen added a comment - Yes, that sounds like a good idea. In fact, it looks like there are many methods in the replications tests' Utils class that could be replaced with methods in PrivilegedFileOpsForTests.
          Hide
          Kathey Marsden added a comment -

          In last night's Windows trunk run I see the following faliure with IBM 1.5:
          1) testReplication_Local_1_Indexing(org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing)java.sql.SQLException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat', see the next exception for details.::SQLSTATE: XBM0AThe database directory 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again.
          at org.apache.derby.client.am.SQLExceptionFactory.getSQLException(Unknown Source)
          at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source)
          at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source)
          at java.sql.DriverManager.getConnection(DriverManager.java:572)
          at java.sql.DriverManager.getConnection(DriverManager.java:218)
          at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.bootMasterDatabase(ReplicationRun.java:868)
          at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.makeReadyForReplication(ReplicationRun.java:2746)
          at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing.testReplication_Local_1_Indexing(ReplicationRun_Local_1Indexing.java:70)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
          at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.runBare(ReplicationRun.java:207)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat', see the next exception for details.::SQLSTATE: XBM0AThe database directory 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again.
          at org.apache.derby.client.am.Connection.completeSqlca(Unknown Source)
          at org.apache.derby.client.net.NetConnectionReply.parseRdbAccessFailed(Unknown Source)
          at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(Unknown Source)
          at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(Unknown Source)
          at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(Unknown Source)
          at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(Unknown Source)
          at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(Unknown Source)
          at org.apache.derby.client.net.NetConnection.flowUSRIDONLconnect(Unknown Source)
          at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source)
          at org.apache.derby.client.net.NetConnection.<init>(Unknown Source)
          at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source)
          ... 35 more

          Is this the same issue, perhaps with a new message or something different?

          Show
          Kathey Marsden added a comment - In last night's Windows trunk run I see the following faliure with IBM 1.5: 1) testReplication_Local_1_Indexing(org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing)java.sql.SQLException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat', see the next exception for details.::SQLSTATE: XBM0AThe database directory 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again. at org.apache.derby.client.am.SQLExceptionFactory.getSQLException(Unknown Source) at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) at org.apache.derby.jdbc.ClientDriver.connect(Unknown Source) at java.sql.DriverManager.getConnection(DriverManager.java:572) at java.sql.DriverManager.getConnection(DriverManager.java:218) at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.bootMasterDatabase(ReplicationRun.java:868) at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.makeReadyForReplication(ReplicationRun.java:2746) at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun_Local_1Indexing.testReplication_Local_1_Indexing(ReplicationRun_Local_1Indexing.java:70) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:79) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112) at org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationRun.runBare(ReplicationRun.java:207) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: XJ041, SQLERRMC: Failed to create database 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat', see the next exception for details.::SQLSTATE: XBM0AThe database directory 'C:\jartest\JarResults.2011-07-04\ibm15_suites.All\db_master\wombat' exists. However, it does not contain the expected 'service.properties' file. Perhaps Derby was brought down in the middle of creating this database. You may want to delete this directory and try creating the database again. at org.apache.derby.client.am.Connection.completeSqlca(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.parseRdbAccessFailed(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.parseAccessRdbError(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.parseACCRDBreply(Unknown Source) at org.apache.derby.client.net.NetConnectionReply.readAccessDatabase(Unknown Source) at org.apache.derby.client.net.NetConnection.readSecurityCheckAndAccessRdb(Unknown Source) at org.apache.derby.client.net.NetConnection.flowSecurityCheckAndAccessRdb(Unknown Source) at org.apache.derby.client.net.NetConnection.flowUSRIDONLconnect(Unknown Source) at org.apache.derby.client.net.NetConnection.flowConnect(Unknown Source) at org.apache.derby.client.net.NetConnection.<init>(Unknown Source) at org.apache.derby.client.net.ClientJDBCObjectFactoryImpl.newNetConnection(Unknown Source) ... 35 more Is this the same issue, perhaps with a new message or something different?
          Hide
          Kristian Waagan added a comment -

          I've seen the same message with the upgrade tests (see DERBY-5297), but in that case I know there was a problem with deleting the database files that's causing it.
          The message was changed in DERBY-4589 (April 13th).

          Show
          Kristian Waagan added a comment - I've seen the same message with the upgrade tests (see DERBY-5297 ), but in that case I know there was a problem with deleting the database files that's causing it. The message was changed in DERBY-4589 (April 13th).
          Hide
          Kristian Waagan added a comment -

          Logged DERBY-5318 to improve the error reporting regarding files that can't be deleted.

          Show
          Kristian Waagan added a comment - Logged DERBY-5318 to improve the error reporting regarding files that can't be deleted.
          Hide
          Myrna van Lunteren added a comment -

          I checked former test results and this failure has not occurred since we moved the vmware machines to a different (faster) server... Closing as cannot reproduce, if it happens again, we can reopen...

          Show
          Myrna van Lunteren added a comment - I checked former test results and this failure has not occurred since we moved the vmware machines to a different (faster) server... Closing as cannot reproduce, if it happens again, we can reopen...
          Show
          Myrna van Lunteren added a comment - This happened again though, in 10.5 testing: http://people.apache.org/~myrnavl/derby_test_results/v10_5/windows/testlog/ibm16/1495656-suites.All_diff.txt

            People

            • Assignee:
              Unassigned
              Reporter:
              Myrna van Lunteren
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development