Uploaded image for project: 'Derby'
  1. Derby
  2. DERBY-5281

Derby DB Corrupt After Crash

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Cannot Reproduce
    • Affects Version/s: 10.8.1.2
    • Fix Version/s: 10.8.1.2
    • Component/s: Store
    • Labels:
      None
    • Environment:
    • Urgency:
      Urgent
    • Bug behavior facts:
      Crash, Data corruption

      Description

      Steps to hit the issue:
      1. Create a derby db with a few tables
      2. Perform some db operations (there's going to be a mix of selects, updates, and inserts here across several tables).
      3. Kill the java process during the db operations
      4. Attempt to start the java process again.
      5. Repeat from (2) a few times.

      After a few iterations, starting derby fails with this exception:

      Exception in thread "main" java.sql.SQLException: Failed to start database '/Users/tim/jiras/mnk-2519/ObjectDataSynchronousWriteTest/objectdb/datadb' with class loader sun.misc.Launcher$AppClassLoader@61e63e3d, see the next exception for details.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source)
      at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source)
      at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source)
      at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source)
      at java.sql.DriverManager.getConnection(DriverManager.java:582)
      at java.sql.DriverManager.getConnection(DriverManager.java:154)
      at OpenDerby.main(OpenDerby.java:16)
      Caused by: java.sql.SQLException: Failed to start database '/Users/tim/jiras/mnk-2519/ObjectDataSynchronousWriteTest/objectdb/datadb' with class loader sun.misc.Launcher$AppClassLoader@61e63e3d, see the next exception for details.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      ... 13 more
      Caused by: java.sql.SQLException: An exception was thrown during transaction abort.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source)
      at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.generateCsSQLException(Unknown Source)
      ... 10 more
      Caused by: ERROR XSTB0: An exception was thrown during transaction abort.
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.store.raw.xact.Xact.abort(Unknown Source)
      at org.apache.derby.impl.store.raw.xact.XactFactory.rollbackAllTransactions(Unknown Source)
      at org.apache.derby.impl.store.raw.log.LogToFile.recover(Unknown Source)
      at org.apache.derby.impl.store.raw.RawStore.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
      at org.apache.derby.impl.store.access.RAMAccessManager.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Unknown Source)
      at org.apache.derby.impl.db.BasicDatabase.bootStore(Unknown Source)
      at org.apache.derby.impl.db.BasicDatabase.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.boot(Unknown Source)
      at org.apache.derby.impl.services.monitor.TopService.bootModule(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(Unknown Source)
      at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(Unknown Source)
      at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Unknown Source)
      ... 10 more
      Caused by: ERROR XSLA8: Cannot rollback transaction 2989, trying to compensate null operation with null
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.store.raw.log.FileLogger.undo(Unknown Source)
      ... 32 more
      Caused by: ERROR XSLA1: Log Record has been sent to the stream, but it cannot be applied to the store (Object null). This may cause recovery problems also.
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.store.raw.log.FileLogger.logAndUndo(Unknown Source)
      at org.apache.derby.impl.store.raw.xact.Xact.logAndUndo(Unknown Source)
      ... 33 more
      Caused by: ERROR XSDB0: Unexpected exception on in-memory page Page(1,Container(0, 1409))
      at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
      at org.apache.derby.impl.store.raw.data.StoredPage.storeRecordForInsert(Unknown Source)
      at org.apache.derby.impl.store.raw.data.StoredPage.storeRecord(Unknown Source)
      at org.apache.derby.impl.store.raw.data.PurgeOperation.undoMe(Unknown Source)
      at org.apache.derby.impl.store.raw.data.PhysicalUndoOperation.doMe(Unknown Source)
      ... 35 more

      The circumstances are essentially the same as what was in DERBY-5248, but the error seems to be a bit different. Opening a fresh issue since the nature of the problem is not quite clear. This issue can be merged if the root cause is found to be the same.

      I'm attaching the corrupted db along with the output from opening the db with a sane build. Please let me know if any more information is needed.

        Attachments

        1. sane_output.zip
          148 kB
          Tim Wu
        2. runtimelogrecorddump.jar
          1.04 MB
          Mike Matrigali
        3. objectdb.zip
          938 kB
          Tim Wu
        4. fulllogrecorddump.jar
          494 kB
          Mike Matrigali
        5. derby.log
          12 kB
          Ulrike
        6. crashed_db_11_18_2011.tar.gz
          1011 kB
          Tim Wu

          Issue Links

            Activity

              People

              • Assignee:
                mikem Mike Matrigali
                Reporter:
                timwu Tim Wu
              • Votes:
                3 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: