Derby
  1. Derby
  2. DERBY-3746

java.sql.SQLException: The exception 'java.lang.NullPointerException' was thrown while evaluating an expression.

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 10.3.1.4
    • Fix Version/s: None
    • Component/s: Store
    • Environment:
      SUSE LINUX 10, Embedded Derby
    • Urgency:
      Normal
    • Bug behavior facts:
      Seen in production

      Description

      Hi

      Following exception occurs while running our application with embedded derby database. The application uses multithreading. This exception occurs while insert query is executed. The insert query is run using JDBC:

      insert into sign (sign, month, day, offset, digest) values (?,?,?,?,?).

      The table for which insert query is fired is:

      create table sign(ID INTEGER NOT NULL GENERATED ALWAYS AS IDENTITY PRIMARY KEY, sign varchar(32), month char(6) not null, day char(2) not null, offset bigint not null, digest varchar(32) for bit data not null)

      There are multiple threads inserting data in database. I tried to search a lot on net but could not find any solution. We're using derby 10.3.1.4. Not sure about the root cause. It will be great if anybody provides some solution because it is creating lot of problems in our application.

      -------------------------------------------------------------------------------------------------------------

      Caused by: java.sql.SQLException: The exception
      'java.lang.NullPointerException' was thrown while evaluating an expression.
      at
      org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
      Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
      at
      org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
      Source)
      at
      org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown
      Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown
      Source)
      at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown
      Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown
      Source)
      at
      org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeStatement(Unknown
      Source)
      at
      org.apache.derby.impl.jdbc.EmbedPreparedStatement.executeUpdate(Unknown Source)
      at
      org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
      at
      org.apache.commons.dbcp.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:101)
      at src.app.database.derby.Derby.insertData(Derby.java:132)
      ... 17 more
      Caused by: java.sql.SQLException: The exception
      'java.lang.NullPointerException' was thrown while evaluating an expression.
      at
      org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at
      org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
      Source)
      ... 30 more
      Caused by: java.sql.SQLException: Java exception: ':
      java.lang.NullPointerException'.
      at
      org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at
      org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown
      Source)
      at
      org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown
      Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source)
      at
      org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown
      Source)
      ... 27 more
      Caused by: java.lang.NullPointerException
      at
      org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown
      Source)
      at
      org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown
      Source)
      at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown
      Source)
      at
      org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown
      Source)
      at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source)
      at
      org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown
      Source)
      at
      org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown
      Source)
      at
      org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSetAutoincrementValue(Unknown
      Source)
      at
      org.apache.derby.impl.sql.execute.InsertResultSet.getSetAutoincrementValue(Unknown
      Source)
      at
      org.apache.derby.impl.sql.execute.BaseActivation.getSetAutoincrementValue(Unknown
      Source)
      at
      org.apache.derby.exe.acd89d8024x0119xf710x52e1x000002ea2ea00.e0(Unknown Source)
      at org.apache.derby.impl.services.reflect.DirectCall.invoke(Unknown
      Source)
      at
      org.apache.derby.impl.sql.execute.RowResultSet.getNextRowCore(Unknown Source)
      at
      org.apache.derby.impl.sql.execute.NormalizeResultSet.getNextRowCore(Unknown
      Source)
      at
      org.apache.derby.impl.sql.execute.DMLWriteResultSet.getNextRowCore(Unknown
      Source)
      at org.apache.derby.impl.sql.execute.InsertResultSet.open(Unknown
      Source)
      at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown
      Source)
      ... 23 more
      ----------------------------------------------------------------------------------

      Also please find the output of sysinfo:

      ------------------ Java Information ------------------

      Java Version: 1.6.0_01

      Java Vendor: Sun Microsystems Inc.

      Java home: C:\Java\jdk1.6.0_01\jre

      Java classpath: .\config;.\lib\activation.jar;.\lib\aperture-1.0.1-beta.jar;
      .\lib\bcmail-jdk14-132.jar;.\lib\bcprov-jdk14-132.jar;.\lib\castor-0.9.3.21-xml.jar;
      .\lib\checkstyle-all-4.2.jar;.\lib\cjkanalyzer.jar;.\lib\commons-beanutils.jar;
      .\lib\commons-collections-3.0.jar;.\lib\commons-dbcp-1.2.1.jar;
      .\lib\commons-digester.jar;.\lib\commons-fileupload.jar;.\lib\commons-lang-2.0.jar;
      .\lib\commons-logging.jar;.\lib\commons-net-1.4.1.jar;.\lib\commons-pool-1.2.jar;
      .\lib\commons-validator.jar;.\lib\app.jar;.\lib\app_13.jar;.\lib\appsync.jar;
      .\lib\derby.jar;.\lib\derbyclient.jar;.\lib\derbynet.jar;.\lib\derbytools.jar;
      .\lib\displaytag-1.0-rc1.jar;.\lib\FontBox-0.1.0-dev.jar;
      .\lib\jakarta-regexp-1.2.jar;.\lib\javaee.jar;.\lib\jcharset.jar;
      .\lib\jcifs-1.2.17.jar;.\lib\jcommon-0.9.6.jar;.\lib\jfreechart-0.9.21.jar;
      .\lib\jtnef.jar;.\lib\junit.jar;.\lib\ldap.jar;.\lib\log4j-1.2.8.jar;
      .\lib\lucene-core-2.2.0.jar;.\lib\lucene-highlighter.jar;.\lib\mail.jar;
      .\lib\mysql-connector-java-3.0.9-stable-bin.jar;
      .\lib\openrdf-rdf2go-2.0.0-beta6.jar;.\lib\openrdf-sesame-2.0-beta6-onejar.jar;
      .\lib\org.semweb4j.rdf2go.api_4.4.7.jar;
      .\lib\org.semweb4j.rdf2go.impl.base_4.4.6.jar;.\lib\pd4ml.jar;
      .\lib\PDFBox-0.7.3.jar;.\lib\poi-3.0.1-FINAL-20070705.jar;.\lib\quartz-1.5.2.jar;
      .\lib\slf4j-api-1.3.0.jar;.\lib\slf4j-simple-1.3.0.jar;.\lib\SNMP4J.jar;
      .\lib\ss_css2.jar;.\lib\struts.jar;.\lib\textmining_v1.jar;
      .\lib\tm-extractors-0.4.jar;.\lib\truezip-6.jar;.\lib\xerces-2_6_2.jar;
      .\lib\windows\FPLibrary.jar

      OS name: Windows XP

      OS architecture: x86

      OS version: 5.1

      Java user name: vibhuti

      Java user home: C:\Documents and Settings\vibhuti

      Java user dir: D:\projects\src\app\dist\app

      java.specification.name: Java Platform API Specification

      java.specification.version: 1.6

      --------- Derby Information --------

      JRE - JDBC: Java SE 6 - JDBC 4.0

      [D:\projects\src\app\dist\app\lib\derby.jar] 10.3.1.4 - (561794)

      [D:\projects\src\app\dist\app\lib\derbytools.jar] 10.3.1.4 - (5617

      94)

      [D:\projects\src\app\dist\app\lib\derbynet.jar] 10.3.1.4 - (561794

      )

      [D:\projects\src\app\dist\app\lib\derbyclient.jar] 10.3.1.4 - (561

      794)

      ------------------------------------------------------

      ----------------- Locale Information -----------------

      ------------------------------------------------------

      1. mailsdb_derby.log
        35 kB
        Lily Wei
      2. mailjdbc_patch022410.diff
        30 kB
        Lily Wei
      3. mailjdbc_patch.diff
        24 kB
        Lily Wei
      4. derby.log
        258 kB
        Lily Wei
      5. derby_mailsdb_XSDG9.log
        204 kB
        Lily Wei

        Issue Links

          Activity

          Hide
          Kristian Waagan added a comment -

          Broke very long classpath into pieces to fix formatting.

          Show
          Kristian Waagan added a comment - Broke very long classpath into pieces to fix formatting.
          Hide
          Stan Bradbury added a comment -

          Please try your testing using the 10.3 fixpack. A very serious bug involving container (e.g. file) handling was fixed: DERBY-3347. The bug produced a number of errors - many of which caused corruption. Please run a consistency check on your database to insure no corrupt information was written to disk. The NPE you experience did occur in container handling code though I am not familiar enough with the code to know if this sort of error could have been caused by DERBY-3347.

          First exeception on the stack you reported : openContainer()
          Caused by: java.lang.NullPointerException
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source)
          at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown

          From Release Notes - DERBY-3347:

          A bug that could cause unrecoverable database corruption has been fixed.
          Symptoms Seen by Applications Affected by Change

          A bug that could cause database corruption was introduced in the 10.3 codeline and affects the following releases:

          • Apache Derby 10.3.1.4
          • Apache Derby 10.3.2.1

          Users who are hit by this bug may experience exceptions at various times during execution of SQL statements, booting or shutdown of a database, or during checkpointing. It may result in a number of different error messages, including any of these:

          ERROR XSDB3: Container information cannot change once written: was 0, now 80
          ERROR XSDG1: Page Page(1039,Container(0, 5856)) could not be written to disk, please check if disk is full.
          ERROR XSDG2: Invalid checksum on Page Page(0,Container(0, 1313))
          ERROR XSDG3: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer4@1afb0c7 could not be accessed
          ERROR XSLA1: Log Record has been sent to the stream, but it cannot be applied to the store (Object null). This may cause recovery problems also.

          HTH

          Show
          Stan Bradbury added a comment - Please try your testing using the 10.3 fixpack. A very serious bug involving container (e.g. file) handling was fixed: DERBY-3347 . The bug produced a number of errors - many of which caused corruption. Please run a consistency check on your database to insure no corrupt information was written to disk. The NPE you experience did occur in container handling code though I am not familiar enough with the code to know if this sort of error could have been caused by DERBY-3347 . First exeception on the stack you reported : openContainer() Caused by: java.lang.NullPointerException at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown From Release Notes - DERBY-3347 : A bug that could cause unrecoverable database corruption has been fixed. Symptoms Seen by Applications Affected by Change A bug that could cause database corruption was introduced in the 10.3 codeline and affects the following releases: Apache Derby 10.3.1.4 Apache Derby 10.3.2.1 Users who are hit by this bug may experience exceptions at various times during execution of SQL statements, booting or shutdown of a database, or during checkpointing. It may result in a number of different error messages, including any of these: ERROR XSDB3: Container information cannot change once written: was 0, now 80 ERROR XSDG1: Page Page(1039,Container(0, 5856)) could not be written to disk, please check if disk is full. ERROR XSDG2: Invalid checksum on Page Page(0,Container(0, 1313)) ERROR XSDG3: Meta-data for Container org.apache.derby.impl.store.raw.data.RAFContainer4@1afb0c7 could not be accessed ERROR XSLA1: Log Record has been sent to the stream, but it cannot be applied to the store (Object null). This may cause recovery problems also. HTH
          Hide
          vibhuti gupta added a comment -

          Thanks for providing me useful inputs regarding the problem. In our
          application there are mutilple threads performing db operations
          constantly, so updating the jars may address the strange issues we
          face. Also perfomance is very crucial for our application. I'll try
          using the latest fixpack either 10.3.3.0 or 10.4.1.0 and get back with
          the results regarding the errors and performance.

          Show
          vibhuti gupta added a comment - Thanks for providing me useful inputs regarding the problem. In our application there are mutilple threads performing db operations constantly, so updating the jars may address the strange issues we face. Also perfomance is very crucial for our application. I'll try using the latest fixpack either 10.3.3.0 or 10.4.1.0 and get back with the results regarding the errors and performance.
          Hide
          David Sitsky added a comment -

          We have seen this issue at a customer deployment using 10.4.1.3 under Windows - not sure which OS version. Here is the stack trace with line numbers. The root exception is the same as what has been reported in this bug.

          Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45)
          at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:87)
          at org.apache.derby.impl.jdbc.Util.javaException(Util.java:244)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:403)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346)
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2125)
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:571)
          at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)
          at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:80)
          at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:238)
          at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:479)
          at org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(EmbedPooledConnection.java:171)
          at org.apache.derby.jdbc.EmbedPooledConnection.<init>(EmbedPooledConnection.java:113)
          at org.apache.derby.jdbc.Driver30.getNewPooledConnection(Driver30.java:141)
          at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.createPooledConnection(EmbeddedConnectionPoolDataSource.java:129)
          at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.getPooledConnection(EmbeddedConnectionPoolDataSource.java:75)
          at bK.makeObject(SourceFile:163)
          at org.apache.commons.pool.impl.StackObjectPool.borrowObject(StackObjectPool.java:131)
          at gh.a(SourceFile:86)
          at vH.a(SourceFile:101)
          at vH.a(SourceFile:157)
          at FO.b(SourceFile:74)
          ... 40 more

          Caused by: java.lang.NullPointerException
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:629)
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:559)
          at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Xact.java:1283)
          at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(OpenConglomerate.java:863)
          at org.apache.derby.impl.store.access.heap.Heap.open(Heap.java:683)
          at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:479)
          at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:1311)
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(DataDictionaryImpl.java:7907)
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(DataDictionaryImpl.java:1556)
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(DataDictionaryImpl.java:1466)
          at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(GenericLanguageConnectionContext.java:363)
          at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(GenericLanguageConnectionContext.java:345)
          at org.apache.derby.impl.db.BasicDatabase.setupConnection(BasicDatabase.java:309)
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(TransactionResourceImpl.java:188)
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:416)
          ... 55 more

          Show
          David Sitsky added a comment - We have seen this issue at a customer deployment using 10.4.1.3 under Windows - not sure which OS version. Here is the stack trace with line numbers. The root exception is the same as what has been reported in this bug. Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(SQLExceptionFactory.java:45) at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Util.java:87) at org.apache.derby.impl.jdbc.Util.javaException(Util.java:244) at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(TransactionResourceImpl.java:403) at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(TransactionResourceImpl.java:346) at org.apache.derby.impl.jdbc.EmbedConnection.handleException(EmbedConnection.java:2125) at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:571) at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73) at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:80) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:238) at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:479) at org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(EmbedPooledConnection.java:171) at org.apache.derby.jdbc.EmbedPooledConnection.<init>(EmbedPooledConnection.java:113) at org.apache.derby.jdbc.Driver30.getNewPooledConnection(Driver30.java:141) at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.createPooledConnection(EmbeddedConnectionPoolDataSource.java:129) at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.getPooledConnection(EmbeddedConnectionPoolDataSource.java:75) at bK.makeObject(SourceFile:163) at org.apache.commons.pool.impl.StackObjectPool.borrowObject(StackObjectPool.java:131) at gh.a(SourceFile:86) at vH.a(SourceFile:101) at vH.a(SourceFile:157) at FO.b(SourceFile:74) ... 40 more Caused by: java.lang.NullPointerException at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:629) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:559) at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Xact.java:1283) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(OpenConglomerate.java:863) at org.apache.derby.impl.store.access.heap.Heap.open(Heap.java:683) at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:479) at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(RAMTransaction.java:1311) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(DataDictionaryImpl.java:7907) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(DataDictionaryImpl.java:1556) at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(DataDictionaryImpl.java:1466) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(GenericLanguageConnectionContext.java:363) at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(GenericLanguageConnectionContext.java:345) at org.apache.derby.impl.db.BasicDatabase.setupConnection(BasicDatabase.java:309) at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(TransactionResourceImpl.java:188) at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:416) ... 55 more
          Hide
          David Sitsky added a comment -

          Whatever is causing this issue, it appears to be transient in nature. We advised our customer to restart the application, which of course closes and re-opens the Derby database, and apparently the above problem didn't surface this time. So it is not entirely clear what is triggering this condition.

          Show
          David Sitsky added a comment - Whatever is causing this issue, it appears to be transient in nature. We advised our customer to restart the application, which of course closes and re-opens the Derby database, and apparently the above problem didn't surface this time. So it is not entirely clear what is triggering this condition.
          Hide
          Trejkaz added a comment -

          In our case this is being caused by a database stored on a file server.

          At some point in time, the network goes out for a moment, resulting in Derby recording a corruption error. Subsequent queries then receive a NullPointerException as some field is set to null in this case, and then closing and reopening the database makes it work again because the database itself has not actually been corrupted.

          So the problem appears to be twofold:
          1. We're getting a NPE instead of a exception saying something about corruption or an inability to read from the store.
          2. Derby is assuming that a failure reading from the store now guarantees a failure to read from the store later, which is not the case for a file store where the files are on another computer being accessed over the network.

          Show
          Trejkaz added a comment - In our case this is being caused by a database stored on a file server. At some point in time, the network goes out for a moment, resulting in Derby recording a corruption error. Subsequent queries then receive a NullPointerException as some field is set to null in this case, and then closing and reopening the database makes it work again because the database itself has not actually been corrupted. So the problem appears to be twofold: 1. We're getting a NPE instead of a exception saying something about corruption or an inability to read from the store. 2. Derby is assuming that a failure reading from the store now guarantees a failure to read from the store later, which is not the case for a file store where the files are on another computer being accessed over the network.
          Hide
          Kathey Marsden added a comment -

          The documentation says:
          You can specify only databases that are local to the machine on which the JVM is running. NFS file systems on UNIX and remote shared files on Windows (//machine/directory) are not guaranteed to work.
          see:
          http://db.apache.org/derby/docs/dev/devguide/cdevdvlp40350.html

          I think the corruption you are seeing is because of the network access to the files. If that is the case I think this issue can be closed as invalid.

          Show
          Kathey Marsden added a comment - The documentation says: You can specify only databases that are local to the machine on which the JVM is running. NFS file systems on UNIX and remote shared files on Windows (//machine/directory) are not guaranteed to work. see: http://db.apache.org/derby/docs/dev/devguide/cdevdvlp40350.html I think the corruption you are seeing is because of the network access to the files. If that is the case I think this issue can be closed as invalid.
          Hide
          Trejkaz added a comment -

          Surely when the network goes back to normal, the database should work again. It seems silly to have to shut down the database and reopen it just to get Derby back into the correct state again, not to mention that it's more or less impossible to do this under Network Server.

          Show
          Trejkaz added a comment - Surely when the network goes back to normal, the database should work again. It seems silly to have to shut down the database and reopen it just to get Derby back into the correct state again, not to mention that it's more or less impossible to do this under Network Server.
          Hide
          Kathey Marsden added a comment -

          I'm afraid not. Derby cannot guarantee the consistency of the database when accessed over the network. It is a common cause for corruption and should not be done.

          Show
          Kathey Marsden added a comment - I'm afraid not. Derby cannot guarantee the consistency of the database when accessed over the network. It is a common cause for corruption and should not be done.
          Hide
          Trejkaz added a comment -

          You appear to be dodging the issue. In cases where we have seen this, the database is not corrupt, Derby just thinks it is until it is forced to reopen it. When it reopens it, it works fine, so why can't it do this under the covers?

          Show
          Trejkaz added a comment - You appear to be dodging the issue. In cases where we have seen this, the database is not corrupt, Derby just thinks it is until it is forced to reopen it. When it reopens it, it works fine, so why can't it do this under the covers?
          Hide
          Kathey Marsden added a comment -

          Well I guess if I had my preference we would have a way to prevent network access of databases completely or find a way to support it. Unfortunately a feasible solution on either count has not presented itself. I wouldn't want to do anything "under the covers" to promote this bad practice. I am glad to hear your database is not yet corrupt. I hope that is indeed the case.

          Show
          Kathey Marsden added a comment - Well I guess if I had my preference we would have a way to prevent network access of databases completely or find a way to support it. Unfortunately a feasible solution on either count has not presented itself. I wouldn't want to do anything "under the covers" to promote this bad practice. I am glad to hear your database is not yet corrupt. I hope that is indeed the case.
          Hide
          Trejkaz added a comment - - edited

          I don't think this really comes down to network vs. non-network. It should come down to a "usable filesystem", and I haven't yet heard a good definition of what constitutes a "usable filesystem". On the mailing list, it was suggested that reliable sync() and locking were the only requirements. As far as I know CIFS and NFS can do both of these, which would suggest that it's just as safe to use a network as a locally attached disk.

          Networks can go down temporarily, sure, but so can "directly" attached disks (e.g. eSATA, or hot-swappable drives. What about USB and FireWire?)

          So it seems to me this particular problem could very well occur if a directly attached disk disappeared and then later reappeared. It's not really an issue of corruption at all... or so you'd hope, otherwise Derby wouldn't be resilient to a power outage either.

          Show
          Trejkaz added a comment - - edited I don't think this really comes down to network vs. non-network. It should come down to a "usable filesystem", and I haven't yet heard a good definition of what constitutes a "usable filesystem". On the mailing list, it was suggested that reliable sync() and locking were the only requirements. As far as I know CIFS and NFS can do both of these, which would suggest that it's just as safe to use a network as a locally attached disk. Networks can go down temporarily, sure, but so can "directly" attached disks (e.g. eSATA, or hot-swappable drives. What about USB and FireWire?) So it seems to me this particular problem could very well occur if a directly attached disk disappeared and then later reappeared. It's not really an issue of corruption at all... or so you'd hope, otherwise Derby wouldn't be resilient to a power outage either.
          Hide
          David Sitsky added a comment -

          I've seen this issue happen again with 10.4.1.3 under Windows, with the same stack-trace as reported earlier (https://issues.apache.org/jira/browse/DERBY-3746?focusedCommentId=12621320#action_12621320). Despite the discussions on this bug about network filesystems, my incidents happen on a local filesystem with an embedded derby connection - no derby server is involved at all.

          Does anyone have any ideas about this? I know shutting down the program and restarting fixes the issue, but it doesn't look very good for the end-user..

          All I know is there must be some sort of race condition occurring, as it is hard to reproduce this.

          Show
          David Sitsky added a comment - I've seen this issue happen again with 10.4.1.3 under Windows, with the same stack-trace as reported earlier ( https://issues.apache.org/jira/browse/DERBY-3746?focusedCommentId=12621320#action_12621320 ). Despite the discussions on this bug about network filesystems, my incidents happen on a local filesystem with an embedded derby connection - no derby server is involved at all. Does anyone have any ideas about this? I know shutting down the program and restarting fixes the issue, but it doesn't look very good for the end-user.. All I know is there must be some sort of race condition occurring, as it is hard to reproduce this.
          Hide
          David Sitsky added a comment -

          To provide more information - I've attached some more stack traces from the derby.log file itself. In our application, we have multiple threads, each with their own embedded derby connection object potentially accessing the same blob of data concurrently. Its unfortunately very difficult to reproduce this issue.

          ------------ BEGIN SHUTDOWN ERROR STACK -------------

          ERROR XSDG0: Page Page(43,Container(0, 1344)) could not be read from disk.
          at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336)
          at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:690)
          at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190)
          at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295)
          at org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2432)
          at org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2482)
          at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319)
          at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.latchPage(OpenConglomerate.java:294)
          at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.fetch(GenericConglomerateController.java:263)
          at org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCore(IndexRowToBaseRowResultSet.java:389)
          at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:255)
          at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
          at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
          at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:367)
          at Pu.a(SourceFile:305)
          at Pu.a(SourceFile:292)
          at IT.a(SourceFile:118)
          at ya.a(SourceFile:160)
          at IT.b(SourceFile:73)
          at qy.read(SourceFile:291)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
          at java.util.zip.CheckedInputStream.read(CheckedInputStream.java:42)
          at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:205)
          at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:197)
          at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:136)
          at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58)
          at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68)
          at uP.a(SourceFile:94)
          at LT.a(SourceFile:78)
          at Kq.a(SourceFile:55)
          at cE.a(SourceFile:229)
          at com.nuix.util.e.a(SourceFile:93)
          at com.nuix.util.e.read(SourceFile:40)
          at org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:337)
          at org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:523)
          at org.apache.lucene.analysis.standard.StandardTokenizer.next(StandardTokenizer.java:139)
          at org.apache.lucene.analysis.standard.StandardFilter.next(StandardFilter.java:41)
          at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45)
          at NF.next(SourceFile:48)
          at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:79)
          at org.apache.lucene.analysis.LowerCaseFilter.next(LowerCaseFilter.java:33)
          at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45)
          at org.apache.lucene.search.similar.MoreLikeThis.addTermFrequencies(MoreLikeThis.java:836)
          at org.apache.lucene.search.similar.MoreLikeThis.retrieveTerms(MoreLikeThis.java:906)
          at org.apache.lucene.search.similar.MoreLikeThis.retrieveInterestingTerms(MoreLikeThis.java:922)
          at gm.a(SourceFile:245)
          at gm.a(SourceFile:286)
          at Hq.<init>(SourceFile:243)
          at Hq.<init>(SourceFile:230)
          at rq.a(SourceFile:175)
          at rq.doInBackground(SourceFile:147)
          at javax.swing.SwingWorker$1.call(SwingWorker.java:278)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          at java.util.concurrent.FutureTask.run(FutureTask.java:138)
          at javax.swing.SwingWorker.run(SwingWorker.java:317)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)
          Caused by: java.nio.channels.ClosedChannelException
          at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:91)
          at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:616)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:471)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:242)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:212)
          at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:666)
          ... 57 more
          ============= begin nested exception, level (1) ===========
          java.nio.channels.ClosedChannelException
          at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:91)
          at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:616)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:471)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:242)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:212)
          at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:666)
          at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190)
          at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295)
          at org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2432)
          at org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2482)
          at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319)
          at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.latchPage(OpenConglomerate.java:294)
          at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.fetch(GenericConglomerateController.java:263)
          at org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCore(IndexRowToBaseRowResultSet.java:389)
          at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:255)
          at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460)
          at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423)
          at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:367)
          at Pu.a(SourceFile:305)
          at Pu.a(SourceFile:292)
          at IT.a(SourceFile:118)
          at ya.a(SourceFile:160)
          at IT.b(SourceFile:73)
          at qy.read(SourceFile:291)
          at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
          at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
          at java.util.zip.CheckedInputStream.read(CheckedInputStream.java:42)
          at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:205)
          at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:197)
          at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:136)
          at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58)
          at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68)
          at uP.a(SourceFile:94)
          at LT.a(SourceFile:78)
          at Kq.a(SourceFile:55)
          at cE.a(SourceFile:229)
          at com.nuix.util.e.a(SourceFile:93)
          at com.nuix.util.e.read(SourceFile:40)
          at org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:337)
          at org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:523)
          at org.apache.lucene.analysis.standard.StandardTokenizer.next(StandardTokenizer.java:139)
          at org.apache.lucene.analysis.standard.StandardFilter.next(StandardFilter.java:41)
          at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45)
          at NF.next(SourceFile:48)
          at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:79)
          at org.apache.lucene.analysis.LowerCaseFilter.next(LowerCaseFilter.java:33)
          at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45)
          at org.apache.lucene.search.similar.MoreLikeThis.addTermFrequencies(MoreLikeThis.java:836)
          at org.apache.lucene.search.similar.MoreLikeThis.retrieveTerms(MoreLikeThis.java:906)
          at org.apache.lucene.search.similar.MoreLikeThis.retrieveInterestingTerms(MoreLikeThis.java:922)
          at gm.a(SourceFile:245)
          at gm.a(SourceFile:286)
          at Hq.<init>(SourceFile:243)
          at Hq.<init>(SourceFile:230)
          at rq.a(SourceFile:175)
          at rq.doInBackground(SourceFile:147)
          at javax.swing.SwingWorker$1.call(SwingWorker.java:278)
          at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
          at java.util.concurrent.FutureTask.run(FutureTask.java:138)
          at javax.swing.SwingWorker.run(SwingWorker.java:317)
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
          at java.lang.Thread.run(Thread.java:619)
          ============= end nested exception, level (1) ===========

          ------------ END SHUTDOWN ERROR STACK -------------

          Show
          David Sitsky added a comment - To provide more information - I've attached some more stack traces from the derby.log file itself. In our application, we have multiple threads, each with their own embedded derby connection object potentially accessing the same blob of data concurrently. Its unfortunately very difficult to reproduce this issue. ------------ BEGIN SHUTDOWN ERROR STACK ------------- ERROR XSDG0: Page Page(43,Container(0, 1344)) could not be read from disk. at org.apache.derby.iapi.error.StandardException.newException(StandardException.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:690) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295) at org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2432) at org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2482) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.latchPage(OpenConglomerate.java:294) at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.fetch(GenericConglomerateController.java:263) at org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCore(IndexRowToBaseRowResultSet.java:389) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:255) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423) at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:367) at Pu.a(SourceFile:305) at Pu.a(SourceFile:292) at IT.a(SourceFile:118) at ya.a(SourceFile:160) at IT.b(SourceFile:73) at qy.read(SourceFile:291) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read(BufferedInputStream.java:237) at java.util.zip.CheckedInputStream.read(CheckedInputStream.java:42) at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:205) at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:197) at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:136) at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58) at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68) at uP.a(SourceFile:94) at LT.a(SourceFile:78) at Kq.a(SourceFile:55) at cE.a(SourceFile:229) at com.nuix.util.e.a(SourceFile:93) at com.nuix.util.e.read(SourceFile:40) at org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:337) at org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:523) at org.apache.lucene.analysis.standard.StandardTokenizer.next(StandardTokenizer.java:139) at org.apache.lucene.analysis.standard.StandardFilter.next(StandardFilter.java:41) at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45) at NF.next(SourceFile:48) at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:79) at org.apache.lucene.analysis.LowerCaseFilter.next(LowerCaseFilter.java:33) at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45) at org.apache.lucene.search.similar.MoreLikeThis.addTermFrequencies(MoreLikeThis.java:836) at org.apache.lucene.search.similar.MoreLikeThis.retrieveTerms(MoreLikeThis.java:906) at org.apache.lucene.search.similar.MoreLikeThis.retrieveInterestingTerms(MoreLikeThis.java:922) at gm.a(SourceFile:245) at gm.a(SourceFile:286) at Hq.<init>(SourceFile:243) at Hq.<init>(SourceFile:230) at rq.a(SourceFile:175) at rq.doInBackground(SourceFile:147) at javax.swing.SwingWorker$1.call(SwingWorker.java:278) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at javax.swing.SwingWorker.run(SwingWorker.java:317) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) Caused by: java.nio.channels.ClosedChannelException at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:91) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:616) at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:471) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:242) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:212) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:666) ... 57 more ============= begin nested exception, level (1) =========== java.nio.channels.ClosedChannelException at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:91) at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:616) at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContainer4.java:471) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFContainer4.java:242) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContainer4.java:212) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.java:666) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPage.java:190) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:295) at org.apache.derby.impl.store.raw.data.FileContainer.getUserPage(FileContainer.java:2432) at org.apache.derby.impl.store.raw.data.FileContainer.getPage(FileContainer.java:2482) at org.apache.derby.impl.store.raw.data.BaseContainerHandle.getPage(BaseContainerHandle.java:319) at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.latchPage(OpenConglomerate.java:294) at org.apache.derby.impl.store.access.conglomerate.GenericConglomerateController.fetch(GenericConglomerateController.java:263) at org.apache.derby.impl.sql.execute.IndexRowToBaseRowResultSet.getNextRowCore(IndexRowToBaseRowResultSet.java:389) at org.apache.derby.impl.sql.execute.ProjectRestrictResultSet.getNextRowCore(ProjectRestrictResultSet.java:255) at org.apache.derby.impl.sql.execute.BasicNoPutResultSetImpl.getNextRow(BasicNoPutResultSetImpl.java:460) at org.apache.derby.impl.jdbc.EmbedResultSet.movePosition(EmbedResultSet.java:423) at org.apache.derby.impl.jdbc.EmbedResultSet.next(EmbedResultSet.java:367) at Pu.a(SourceFile:305) at Pu.a(SourceFile:292) at IT.a(SourceFile:118) at ya.a(SourceFile:160) at IT.b(SourceFile:73) at qy.read(SourceFile:291) at java.io.BufferedInputStream.fill(BufferedInputStream.java:218) at java.io.BufferedInputStream.read(BufferedInputStream.java:237) at java.util.zip.CheckedInputStream.read(CheckedInputStream.java:42) at java.util.zip.GZIPInputStream.readUByte(GZIPInputStream.java:205) at java.util.zip.GZIPInputStream.readUShort(GZIPInputStream.java:197) at java.util.zip.GZIPInputStream.readHeader(GZIPInputStream.java:136) at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:58) at java.util.zip.GZIPInputStream.<init>(GZIPInputStream.java:68) at uP.a(SourceFile:94) at LT.a(SourceFile:78) at Kq.a(SourceFile:55) at cE.a(SourceFile:229) at com.nuix.util.e.a(SourceFile:93) at com.nuix.util.e.read(SourceFile:40) at org.apache.lucene.analysis.standard.StandardTokenizerImpl.zzRefill(StandardTokenizerImpl.java:337) at org.apache.lucene.analysis.standard.StandardTokenizerImpl.getNextToken(StandardTokenizerImpl.java:523) at org.apache.lucene.analysis.standard.StandardTokenizer.next(StandardTokenizer.java:139) at org.apache.lucene.analysis.standard.StandardFilter.next(StandardFilter.java:41) at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45) at NF.next(SourceFile:48) at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:79) at org.apache.lucene.analysis.LowerCaseFilter.next(LowerCaseFilter.java:33) at org.apache.lucene.analysis.TokenStream.next(TokenStream.java:45) at org.apache.lucene.search.similar.MoreLikeThis.addTermFrequencies(MoreLikeThis.java:836) at org.apache.lucene.search.similar.MoreLikeThis.retrieveTerms(MoreLikeThis.java:906) at org.apache.lucene.search.similar.MoreLikeThis.retrieveInterestingTerms(MoreLikeThis.java:922) at gm.a(SourceFile:245) at gm.a(SourceFile:286) at Hq.<init>(SourceFile:243) at Hq.<init>(SourceFile:230) at rq.a(SourceFile:175) at rq.doInBackground(SourceFile:147) at javax.swing.SwingWorker$1.call(SwingWorker.java:278) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at javax.swing.SwingWorker.run(SwingWorker.java:317) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:619) ============= end nested exception, level (1) =========== ------------ END SHUTDOWN ERROR STACK -------------
          Hide
          Trejkaz added a comment -

          I added some more code to RAFContainer to record the stack of "who" was closing the container. And just now the error occurred for me, here's the stack of the culprit who closed the container:

          java.lang.Error: Closed via closeContainer
          at org.apache.derby.impl.store.raw.data.RAFContainer.closeContainer(RAFContainer.java:203)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.closeContainer(RAFContainer4.java:183)
          at org.apache.derby.impl.store.raw.data.FileContainer.clearIdentity(FileContainer.java:473)
          at org.apache.derby.impl.services.cache.ConcurrentCache.evictEntry(ConcurrentCache.java:188)
          at org.apache.derby.impl.services.cache.ClockPolicy.rotateClock(ClockPolicy.java:448)
          at org.apache.derby.impl.services.cache.ClockPolicy.insertEntry(ClockPolicy.java:176)
          at org.apache.derby.impl.services.cache.ConcurrentCache.insertIntoFreeSlot(ConcurrentCache.java:208)
          at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:284)
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:629)
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openDroppedContainer(BaseDataFileFactory.java:579)
          at org.apache.derby.impl.store.raw.xact.Xact.openDroppedContainer(Xact.java:1305)
          at org.apache.derby.impl.store.raw.data.ContainerBasicOperation.findContainer(ContainerBasicOperation.java:145)
          at org.apache.derby.impl.store.raw.data.ContainerBasicOperation.needsRedo(ContainerBasicOperation.java:213)
          at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395)
          at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920)
          at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334)
          at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553)
          at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)
          at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019)
          at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553)
          at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427)
          at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:780)
          at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:196)
          at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999)
          at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291)
          at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1836)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1702)
          at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1531)
          at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:1001)
          at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550)
          at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2416)
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:355)
          at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73)
          at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:80)
          at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:238)
          at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:479)
          at org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(EmbedPooledConnection.java:171)
          at org.apache.derby.jdbc.EmbedPooledConnection.<init>(EmbedPooledConnection.java:113)
          at org.apache.derby.jdbc.Driver30.getNewPooledConnection(Driver30.java:141)
          at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.createPooledConnection(EmbeddedConnectionPoolDataSource.java:129)
          at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.getPooledConnection(EmbeddedConnectionPoolDataSource.java:75)

          Show
          Trejkaz added a comment - I added some more code to RAFContainer to record the stack of "who" was closing the container. And just now the error occurred for me, here's the stack of the culprit who closed the container: java.lang.Error: Closed via closeContainer at org.apache.derby.impl.store.raw.data.RAFContainer.closeContainer(RAFContainer.java:203) at org.apache.derby.impl.store.raw.data.RAFContainer4.closeContainer(RAFContainer4.java:183) at org.apache.derby.impl.store.raw.data.FileContainer.clearIdentity(FileContainer.java:473) at org.apache.derby.impl.services.cache.ConcurrentCache.evictEntry(ConcurrentCache.java:188) at org.apache.derby.impl.services.cache.ClockPolicy.rotateClock(ClockPolicy.java:448) at org.apache.derby.impl.services.cache.ClockPolicy.insertEntry(ClockPolicy.java:176) at org.apache.derby.impl.services.cache.ConcurrentCache.insertIntoFreeSlot(ConcurrentCache.java:208) at org.apache.derby.impl.services.cache.ConcurrentCache.find(ConcurrentCache.java:284) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(BaseDataFileFactory.java:629) at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openDroppedContainer(BaseDataFileFactory.java:579) at org.apache.derby.impl.store.raw.xact.Xact.openDroppedContainer(Xact.java:1305) at org.apache.derby.impl.store.raw.data.ContainerBasicOperation.findContainer(ContainerBasicOperation.java:145) at org.apache.derby.impl.store.raw.data.ContainerBasicOperation.needsRedo(ContainerBasicOperation.java:213) at org.apache.derby.impl.store.raw.log.FileLogger.redo(FileLogger.java:1395) at org.apache.derby.impl.store.raw.log.LogToFile.recover(LogToFile.java:920) at org.apache.derby.impl.store.raw.RawStore.boot(RawStore.java:334) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.store.access.RAMAccessManager.boot(RAMAccessManager.java:1019) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.startModule(BaseMonitor.java:553) at org.apache.derby.iapi.services.monitor.Monitor.bootServiceModule(Monitor.java:427) at org.apache.derby.impl.db.BasicDatabase.bootStore(BasicDatabase.java:780) at org.apache.derby.impl.db.BasicDatabase.boot(BasicDatabase.java:196) at org.apache.derby.impl.services.monitor.BaseMonitor.boot(BaseMonitor.java:1999) at org.apache.derby.impl.services.monitor.TopService.bootModule(TopService.java:291) at org.apache.derby.impl.services.monitor.BaseMonitor.bootService(BaseMonitor.java:1836) at org.apache.derby.impl.services.monitor.BaseMonitor.startProviderService(BaseMonitor.java:1702) at org.apache.derby.impl.services.monitor.BaseMonitor.findProviderAndStartService(BaseMonitor.java:1531) at org.apache.derby.impl.services.monitor.BaseMonitor.startPersistentService(BaseMonitor.java:1001) at org.apache.derby.iapi.services.monitor.Monitor.startPersistentService(Monitor.java:550) at org.apache.derby.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:2416) at org.apache.derby.impl.jdbc.EmbedConnection.<init>(EmbedConnection.java:355) at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(EmbedConnection30.java:73) at org.apache.derby.jdbc.Driver30.getNewEmbedConnection(Driver30.java:80) at org.apache.derby.jdbc.InternalDriver.connect(InternalDriver.java:238) at org.apache.derby.jdbc.EmbeddedDataSource.getConnection(EmbeddedDataSource.java:479) at org.apache.derby.jdbc.EmbedPooledConnection.openRealConnection(EmbedPooledConnection.java:171) at org.apache.derby.jdbc.EmbedPooledConnection.<init>(EmbedPooledConnection.java:113) at org.apache.derby.jdbc.Driver30.getNewPooledConnection(Driver30.java:141) at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.createPooledConnection(EmbeddedConnectionPoolDataSource.java:129) at org.apache.derby.jdbc.EmbeddedConnectionPoolDataSource.getPooledConnection(EmbeddedConnectionPoolDataSource.java:75)
          Hide
          Kathey Marsden added a comment -

          Is this still an issue? If so, any luck on getting a reproduction?

          Show
          Kathey Marsden added a comment - Is this still an issue? If so, any luck on getting a reproduction?
          Hide
          David Sitsky added a comment -

          We gave up using RAFContainer4 due to the intermittent channel closed exceptions. We modified the derby code to use the regular RAFContainer which just uses input/output streams. Since that change, we haven't had any issues, and performance was pretty much the same anyway.

          Its not easy to reproduce... but there must be some subtle issue with the way channels are used / closed under Win32.

          Show
          David Sitsky added a comment - We gave up using RAFContainer4 due to the intermittent channel closed exceptions. We modified the derby code to use the regular RAFContainer which just uses input/output streams. Since that change, we haven't had any issues, and performance was pretty much the same anyway. Its not easy to reproduce... but there must be some subtle issue with the way channels are used / closed under Win32.
          Hide
          Knut Anders Hatlen added a comment -

          Triaged for 10.5.2.

          NPE on the same code location has also been reported in DERBY-3524 and DERBY-4167.

          If the stack trace above really is the culprit, it means that we evict an item from the container cache while it's still in use. Since items are evicted, I assume the number of tables/indexes in the query mix is fairly high. Default container cache size is 100.

          Show
          Knut Anders Hatlen added a comment - Triaged for 10.5.2. NPE on the same code location has also been reported in DERBY-3524 and DERBY-4167 . If the stack trace above really is the culprit, it means that we evict an item from the container cache while it's still in use. Since items are evicted, I assume the number of tables/indexes in the query mix is fairly high. Default container cache size is 100.
          Hide
          Matthew Cooper added a comment -

          I just got this for the first time - no particular cause that I can see. Restarting the JVM fixed it.

          So does "Triaged for 10.5.2." mean it is going to get fixed in that version - and any ideas when that is likely to be?

          (Oh and Hi to Sitsky - small world eh?)

          23-Feb-2010 11:55:32.815 WARN pool0-4540 com.eventzero.occurrence.service.rule.RuleService | Unexpected exception in rule service
          java.lang.RuntimeException: Could not create connection for [ConnectionPool@24df90e8 driverClassName=org.apache.derby.jdbc.EmbeddedDriver username=XXXXXX]
          at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:205) [common.jar:v3.6.6_05]
          at com.eventzero.util.jdbc.ConnectionPool.take(ConnectionPool.java:125) [common.jar:v3.6.6_05]
          at com.eventzero.occurrence.jdbc.AbstractDao.getConnection(AbstractDao.java:167) [corlog-core.jar:v3.6.6_05]
          at com.eventzero.summary.dao.SummaryDAO.query(SummaryDAO.java:1010) [occurrence-summary-engine.jar:v3.6.6_05]
          at com.eventzero.summary.Query.getEvents(Query.java:42) [occurrence-summary-engine.jar:v3.6.6_05]
          at com.eventzero.summary.Rule.query(Rule.java:270) [occurrence-summary-engine.jar:v3.6.6_05]
          at com.eventzero.summary.RuleManager.query(RuleManager.java:140) [occurrence-summary-engine.jar:v3.6.6_05]
          at com.eventzero.summary.SummaryEngine.query(SummaryEngine.java:260) [occurrence-summary-engine.jar:v3.6.6_05]
          at com.eventzero.occurrence.service.rule.RuleService$SummaryEngineQueryRuleController.start(RuleService.java:533) [corlog.jar:v3.6.6_05]
          at com.eventzero.occurrence.service.rule.RuleService$Rule$1.run(RuleService.java:806) [corlog.jar:v3.6.6_05]
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_14]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_14]
          at java.lang.Thread.run(Thread.java:619) [na:1.6.0_14]
          Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) [derby-10.5.1.1.jar:na]
          at java.sql.DriverManager.getConnection(DriverManager.java:582) [na:1.6.0_14]
          at java.sql.DriverManager.getConnection(DriverManager.java:185) [na:1.6.0_14]
          at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:201) [common.jar:v3.6.6_05]
          ... 12 common frames omitted
          Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Java exception: ': java.lang.NullPointerException'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) [derby-10.5.1.1.jar:na]
          ... 27 common frames omitted
          Caused by: java.lang.NullPointerException: null
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMinion(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown Source) [derby-10.5.1.1.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown Source) [derby-10.5.1.1.jar:na]
          ... 21 common frames omitted

          Show
          Matthew Cooper added a comment - I just got this for the first time - no particular cause that I can see. Restarting the JVM fixed it. So does "Triaged for 10.5.2." mean it is going to get fixed in that version - and any ideas when that is likely to be? (Oh and Hi to Sitsky - small world eh?) 23-Feb-2010 11:55:32.815 WARN pool0-4540 com.eventzero.occurrence.service.rule.RuleService | Unexpected exception in rule service java.lang.RuntimeException: Could not create connection for [ConnectionPool@24df90e8 driverClassName=org.apache.derby.jdbc.EmbeddedDriver username=XXXXXX] at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:205) [common.jar:v3.6.6_05] at com.eventzero.util.jdbc.ConnectionPool.take(ConnectionPool.java:125) [common.jar:v3.6.6_05] at com.eventzero.occurrence.jdbc.AbstractDao.getConnection(AbstractDao.java:167) [corlog-core.jar:v3.6.6_05] at com.eventzero.summary.dao.SummaryDAO.query(SummaryDAO.java:1010) [occurrence-summary-engine.jar:v3.6.6_05] at com.eventzero.summary.Query.getEvents(Query.java:42) [occurrence-summary-engine.jar:v3.6.6_05] at com.eventzero.summary.Rule.query(Rule.java:270) [occurrence-summary-engine.jar:v3.6.6_05] at com.eventzero.summary.RuleManager.query(RuleManager.java:140) [occurrence-summary-engine.jar:v3.6.6_05] at com.eventzero.summary.SummaryEngine.query(SummaryEngine.java:260) [occurrence-summary-engine.jar:v3.6.6_05] at com.eventzero.occurrence.service.rule.RuleService$SummaryEngineQueryRuleController.start(RuleService.java:533) [corlog.jar:v3.6.6_05] at com.eventzero.occurrence.service.rule.RuleService$Rule$1.run(RuleService.java:806) [corlog.jar:v3.6.6_05] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_14] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_14] at java.lang.Thread.run(Thread.java:619) [na:1.6.0_14] Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) [derby-10.5.1.1.jar:na] at java.sql.DriverManager.getConnection(DriverManager.java:582) [na:1.6.0_14] at java.sql.DriverManager.getConnection(DriverManager.java:185) [na:1.6.0_14] at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:201) [common.jar:v3.6.6_05] ... 12 common frames omitted Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) [derby-10.5.1.1.jar:na] ... 27 common frames omitted Caused by: java.lang.NullPointerException: null at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMinion(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown Source) [derby-10.5.1.1.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown Source) [derby-10.5.1.1.jar:na] ... 21 common frames omitted
          Hide
          Kristian Waagan added a comment -

          Hi Matthew,

          I see that you're hitting the NPE during connect, and have a few questions:

          • are you running embedded only?
          • do you have multiple threads accessing the database, specifically shutting down and booting/connecting "at the same time"?
          • when you hit the problem, did it persist consistently (i.e. it happened on every subsequent connection attempt)?

          Thanks,

          Show
          Kristian Waagan added a comment - Hi Matthew, I see that you're hitting the NPE during connect, and have a few questions: are you running embedded only? do you have multiple threads accessing the database, specifically shutting down and booting/connecting "at the same time"? when you hit the problem, did it persist consistently (i.e. it happened on every subsequent connection attempt)? Thanks,
          Hide
          Knut Anders Hatlen added a comment -

          > So does "Triaged for 10.5.2." mean it is going to get fixed in that version - and any ideas when that is likely to be?

          It just means that we reviewed it in the 10.5.2 bug triage and made sure that the bug meta data (component, priority, urgency, affect version, etc...) had sensible values.

          Show
          Knut Anders Hatlen added a comment - > So does "Triaged for 10.5.2." mean it is going to get fixed in that version - and any ideas when that is likely to be? It just means that we reviewed it in the 10.5.2 bug triage and made sure that the bug meta data (component, priority, urgency, affect version, etc...) had sensible values.
          Hide
          Lily Wei added a comment -

          While modify system/mailjdbc test, I encounter this bug. Please see the mailsdb_derby.log and https://issues.apache.org/jira/browse/DERBY-4166?focusedCommentId=12735394&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12735394.
          After running the test for three hours, I got exception like:
          ------------ BEGIN SHUTDOWN ERROR STACK -------------

          ERROR XSDG0: Page Page(4,Container(0, 144)) could not be read from disk.^M
          at org.apache.derby.iapi.error.StandardException.newException(StandardEx
          ception.java:336)
          at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.j
          ava:695)
          at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPag
          e.java:190)
          ...
          ============= begin nested exception, level (1) ===========
          java.nio.channels.ClosedChannelException
          at sun.nio.ch.FileChannelImpl.ensureOpen(Unknown Source)
          at sun.nio.ch.FileChannelImpl.read(Unknown Source)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContai
          ner4.java:482)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFConta
          iner4.java:244)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContai
          ner4.java:214)

          To reproduce this issue, please apply the patch (mailjdbc_patch.diff) for mailjdbc test and run it with embedded mode for more than 3 hours.

          Show
          Lily Wei added a comment - While modify system/mailjdbc test, I encounter this bug. Please see the mailsdb_derby.log and https://issues.apache.org/jira/browse/DERBY-4166?focusedCommentId=12735394&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12735394 . After running the test for three hours, I got exception like: ------------ BEGIN SHUTDOWN ERROR STACK ------------- ERROR XSDG0: Page Page(4,Container(0, 144)) could not be read from disk.^M at org.apache.derby.iapi.error.StandardException.newException(StandardEx ception.java:336) at org.apache.derby.impl.store.raw.data.CachedPage.readPage(CachedPage.j ava:695) at org.apache.derby.impl.store.raw.data.CachedPage.setIdentity(CachedPag e.java:190) ... ============= begin nested exception, level (1) =========== java.nio.channels.ClosedChannelException at sun.nio.ch.FileChannelImpl.ensureOpen(Unknown Source) at sun.nio.ch.FileChannelImpl.read(Unknown Source) at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContai ner4.java:482) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFConta iner4.java:244) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage(RAFContai ner4.java:214) To reproduce this issue, please apply the patch (mailjdbc_patch.diff) for mailjdbc test and run it with embedded mode for more than 3 hours.
          Hide
          Kathey Marsden added a comment -

          Hi Lily,

          Mike mentioned to me today that the ClosedChannelException might just be the result of an interrupted thread in the test. Do we still do a thread interrupt anywhere in mailjdbc?

          Show
          Kathey Marsden added a comment - Hi Lily, Mike mentioned to me today that the ClosedChannelException might just be the result of an interrupted thread in the test. Do we still do a thread interrupt anywhere in mailjdbc?
          Hide
          Lily Wei added a comment -

          Hi There:
          Yes, the mailjdbc Refresh thread will be interrupted by Browse thread if it is finishing the doWork() method and the thread is sleeping. I also see XSDG9: Derby thread received an interrupt during a disk I/O operation.
          ERROR XSDG9: Derby thread received an interrupt during a disk I/O operation, ple
          ase check your application for the source of the interrupt.
          at org.apache.derby.iapi.error.StandardException.newException(StandardEx
          ception.java:279)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContai
          ner4.java:489)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFConta
          iner4.java:244)

          The derby.log and mailjdbc_patch022410.diff is attached.
          The mailjdbc_patch022410.diff include take out select from Refresh.java, change the browse mail to read message and read attach message. Add more sleep between thread to avoid deadlock.

          Show
          Lily Wei added a comment - Hi There: Yes, the mailjdbc Refresh thread will be interrupted by Browse thread if it is finishing the doWork() method and the thread is sleeping. I also see XSDG9: Derby thread received an interrupt during a disk I/O operation. ERROR XSDG9: Derby thread received an interrupt during a disk I/O operation, ple ase check your application for the source of the interrupt. at org.apache.derby.iapi.error.StandardException.newException(StandardEx ception.java:279) at org.apache.derby.impl.store.raw.data.RAFContainer4.readFull(RAFContai ner4.java:489) at org.apache.derby.impl.store.raw.data.RAFContainer4.readPage0(RAFConta iner4.java:244) The derby.log and mailjdbc_patch022410.diff is attached. The mailjdbc_patch022410.diff include take out select from Refresh.java, change the browse mail to read message and read attach message. Add more sleep between thread to avoid deadlock.
          Hide
          Knut Anders Hatlen added a comment -

          Judging by the line numbers in David's stack trace (from Derby 10.4.1.3), the NullPointerException is thrown on this line in BaseDataFileFactory.openContainer():

          // see if the container exists
          FileContainer container = (FileContainer) containerCache.find(identity);

          So it must be containerCache that's null. The only code I find that sets containerCache to null is the method markCorrupt(), which is typically called when Derby's store module encounters an IOException, or some other error that Derby doesn't know how to recover from except by rebooting the database. The exception that causes markCorrupt() to be called should be re-thrown, so I would have expected that to be present in derby.log too.

          Lily, it wasn't quite clear to me from your comments whether you saw a NullPointerException in addition to the ClosedChannelException, or if you only saw the ClosedChannelException. Do you remember? It looks like a ClosedChannelException at that point in the code will cause a call to markCorrupt(), which again could make other threads accessing the same database fail with a NullPointerException.

          Show
          Knut Anders Hatlen added a comment - Judging by the line numbers in David's stack trace (from Derby 10.4.1.3), the NullPointerException is thrown on this line in BaseDataFileFactory.openContainer(): // see if the container exists FileContainer container = (FileContainer) containerCache.find(identity); So it must be containerCache that's null. The only code I find that sets containerCache to null is the method markCorrupt(), which is typically called when Derby's store module encounters an IOException, or some other error that Derby doesn't know how to recover from except by rebooting the database. The exception that causes markCorrupt() to be called should be re-thrown, so I would have expected that to be present in derby.log too. Lily, it wasn't quite clear to me from your comments whether you saw a NullPointerException in addition to the ClosedChannelException, or if you only saw the ClosedChannelException. Do you remember? It looks like a ClosedChannelException at that point in the code will cause a call to markCorrupt(), which again could make other threads accessing the same database fail with a NullPointerException.
          Hide
          Knut Anders Hatlen added a comment -

          I've linked this issue to DERBY-4741 since interrupts that make I/O operations fail is a possible cause for the NullPointerExceptions.

          Show
          Knut Anders Hatlen added a comment - I've linked this issue to DERBY-4741 since interrupts that make I/O operations fail is a possible cause for the NullPointerExceptions.
          Hide
          Lily Wei added a comment -

          Hi Knut:
          Thank you so much for looking at the issue. This will be a good one to get fix. I remember I saw a lot of strange behavior after the ClosedChannelException, NullPointerException can be one of them. I did not capture other exception because I thought we might be too late to fix other exception. I was assuming the root cause is the ClosedChannelException. As you can see in the log, we got more than one ClosedChannelException and the tests is calling a lot of interrupt which can call makeCorrupt(). Is there something such as narrow the test case done we can do to help? Thanks!

          Show
          Lily Wei added a comment - Hi Knut: Thank you so much for looking at the issue. This will be a good one to get fix. I remember I saw a lot of strange behavior after the ClosedChannelException, NullPointerException can be one of them. I did not capture other exception because I thought we might be too late to fix other exception. I was assuming the root cause is the ClosedChannelException. As you can see in the log, we got more than one ClosedChannelException and the tests is calling a lot of interrupt which can call makeCorrupt(). Is there something such as narrow the test case done we can do to help? Thanks!
          Hide
          Lily Wei added a comment -

          When testing 10.6.2.0 with mailjdbc test, I experienced this issue again. Attach derby.log for reference. I reproduced it by running mailjdbc more than 30 hours with embedded engine.

          Show
          Lily Wei added a comment - When testing 10.6.2.0 with mailjdbc test, I experienced this issue again. Attach derby.log for reference. I reproduced it by running mailjdbc more than 30 hours with embedded engine.
          Hide
          Kathey Marsden added a comment -

          Lily, I don't actually see a NullPointerException in the derby.log. It looks like the interrupts in the test cause the ClosedChannelExceptions but I seem to be missing the NullPointerException.

          Show
          Kathey Marsden added a comment - Lily, I don't actually see a NullPointerException in the derby.log. It looks like the interrupts in the test cause the ClosedChannelExceptions but I seem to be missing the NullPointerException.
          Hide
          Lily Wei added a comment -

          Hi Kathey:
          Yes, there is no NullPointerException in the derby.log. I am seeing XSDG0: Page Page(626,Container(0, 1136)) could not be read from disk and CloseChannelException after running 30 hours of mailjdbc test. These can very well cause by interrupts in the test. If the CloseChannelException I am seeing from mailjdbc test does not relate to NullPointerException while evaluating an expression. I am sorry for the confusion. Thanks Lily.

          Show
          Lily Wei added a comment - Hi Kathey: Yes, there is no NullPointerException in the derby.log. I am seeing XSDG0: Page Page(626,Container(0, 1136)) could not be read from disk and CloseChannelException after running 30 hours of mailjdbc test. These can very well cause by interrupts in the test. If the CloseChannelException I am seeing from mailjdbc test does not relate to NullPointerException while evaluating an expression. I am sorry for the confusion. Thanks Lily.
          Hide
          Mike Matrigali added a comment -

          triage for derby 10.9. I think we should close this issue as can not reproduce. From analysis of the the stack traces I believe the original user reported
          problem is likely fixed by the interrupt work done in 10.8, in the linked issue DERBY-4741.

          Show
          Mike Matrigali added a comment - triage for derby 10.9. I think we should close this issue as can not reproduce. From analysis of the the stack traces I believe the original user reported problem is likely fixed by the interrupt work done in 10.8, in the linked issue DERBY-4741 .
          Hide
          Matthew Cooper added a comment -

          Problem is not fixed in 10.8.1.2.

          Exact same stack trace as I last listed, occurred this morning and 2 days ago. Restart of service fixed the issue both times. Regarding location of database, it is on local filesystem on a virtual machine.

          To answer Kristian's questions (sorry, I didn't receive mail notifications for comments):

          • are you running embedded only?
            • Embedded but running network server on localhost for other processes to access
          • do you have multiple threads accessing the database, specifically shutting down and booting/connecting "at the same time"?
            • There are multiple threads accessing the database, but the database is 'booted' before these threads start.
          • when you hit the problem, did it persist consistently (i.e. it happened on every subsequent connection attempt)?
            • Yes, until the process is restarted.

          To repeat, this is the stack trace we are getting.

          23-Feb-2012 04:36:07.788 WARN 2031264-17 com.eventzero.occurrence.service.rule.RuleService | Unexpected exception in rule service
          java.lang.RuntimeException: Could not create connection for [ConnectionPool@2fd09539 driverClassName=org.apache.derby.jdbc.EmbeddedDriver username=CORLOG]
          at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:205) [common-e7817d5958e473be2275283ab647b8ca.jar:v3.6.6_29]
          at com.eventzero.util.jdbc.ConnectionPool.take(ConnectionPool.java:125) [common-e7817d5958e473be2275283ab647b8ca.jar:v3.6.6_29]
          at com.eventzero.occurrence.jdbc.AbstractDao.getConnection(AbstractDao.java:169) [corlog-core-64e317f6de56715c87e74d9a225d7c20.jar:v3.6.6_29]
          at com.eventzero.summary.dao.SummaryDAO.query(SummaryDAO.java:359) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29]
          at com.eventzero.summary.query.Query.getEvents(Query.java:60) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29]
          at com.eventzero.summary.Rule.query(Rule.java:394) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29]
          at com.eventzero.summary.RuleManager.query(RuleManager.java:170) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29]
          at com.eventzero.summary.SummaryEngine.query(SummaryEngine.java:306) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29]
          at com.eventzero.occurrence.service.rule.RuleService$SummaryEngineQueryRuleController.start(RuleService.java:577) [corlog-7b41983585314582e052f288c5a31a6f.jar:v3.6.6_29]
          at com.eventzero.occurrence.service.rule.RuleService$Rule$1.run(RuleService.java:851) [corlog-7b41983585314582e052f288c5a31a6f.jar:v3.6.6_29]
          at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_17]
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_17]
          at java.lang.Thread.run(Thread.java:619) [na:1.6.0_17]
          Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) [derby-10.8.1.2.jar:na]
          at java.sql.DriverManager.getConnection(DriverManager.java:582) [na:1.6.0_17]
          at java.sql.DriverManager.getConnection(DriverManager.java:185) [na:1.6.0_17]
          at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:201) [common-e7817d5958e473be2275283ab647b8ca.jar:v3.6.6_29]
          ... 12 common frames omitted
          Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Java exception: ': java.lang.NullPointerException'.
          at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) [derby-10.8.1.2.jar:na]
          ... 27 common frames omitted
          Caused by: java.lang.NullPointerException: null
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMinion(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown Source) [derby-10.8.1.2.jar:na]
          at org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown Source) [derby-10.8.1.2.jar:na]
          ... 21 common frames omitted

          Show
          Matthew Cooper added a comment - Problem is not fixed in 10.8.1.2. Exact same stack trace as I last listed, occurred this morning and 2 days ago. Restart of service fixed the issue both times. Regarding location of database, it is on local filesystem on a virtual machine. To answer Kristian's questions (sorry, I didn't receive mail notifications for comments): are you running embedded only? Embedded but running network server on localhost for other processes to access do you have multiple threads accessing the database, specifically shutting down and booting/connecting "at the same time"? There are multiple threads accessing the database, but the database is 'booted' before these threads start. when you hit the problem, did it persist consistently (i.e. it happened on every subsequent connection attempt)? Yes, until the process is restarted. To repeat, this is the stack trace we are getting. 23-Feb-2012 04:36:07.788 WARN 2031264-17 com.eventzero.occurrence.service.rule.RuleService | Unexpected exception in rule service java.lang.RuntimeException: Could not create connection for [ConnectionPool@2fd09539 driverClassName=org.apache.derby.jdbc.EmbeddedDriver username=CORLOG] at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:205) [common-e7817d5958e473be2275283ab647b8ca.jar:v3.6.6_29] at com.eventzero.util.jdbc.ConnectionPool.take(ConnectionPool.java:125) [common-e7817d5958e473be2275283ab647b8ca.jar:v3.6.6_29] at com.eventzero.occurrence.jdbc.AbstractDao.getConnection(AbstractDao.java:169) [corlog-core-64e317f6de56715c87e74d9a225d7c20.jar:v3.6.6_29] at com.eventzero.summary.dao.SummaryDAO.query(SummaryDAO.java:359) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29] at com.eventzero.summary.query.Query.getEvents(Query.java:60) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29] at com.eventzero.summary.Rule.query(Rule.java:394) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29] at com.eventzero.summary.RuleManager.query(RuleManager.java:170) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29] at com.eventzero.summary.SummaryEngine.query(SummaryEngine.java:306) [occurrence-summary-engine-efdda245967ddcae7aa0b6c8902c36c5.jar:v3.6.6_29] at com.eventzero.occurrence.service.rule.RuleService$SummaryEngineQueryRuleController.start(RuleService.java:577) [corlog-7b41983585314582e052f288c5a31a6f.jar:v3.6.6_29] at com.eventzero.occurrence.service.rule.RuleService$Rule$1.run(RuleService.java:851) [corlog-7b41983585314582e052f288c5a31a6f.jar:v3.6.6_29] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [na:1.6.0_17] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [na:1.6.0_17] at java.lang.Thread.run(Thread.java:619) [na:1.6.0_17] Caused by: java.sql.SQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory40.getSQLException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.Util.javaException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.<init>(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection30.<init>(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection40.<init>(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.jdbc.Driver40.getNewEmbedConnection(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.jdbc.InternalDriver.connect(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.jdbc.AutoloadedDriver.connect(Unknown Source) [derby-10.8.1.2.jar:na] at java.sql.DriverManager.getConnection(DriverManager.java:582) [na:1.6.0_17] at java.sql.DriverManager.getConnection(DriverManager.java:185) [na:1.6.0_17] at com.eventzero.util.jdbc.ConnectionPool.createConnection(ConnectionPool.java:201) [common-e7817d5958e473be2275283ab647b8ca.jar:v3.6.6_29] ... 12 common frames omitted Caused by: org.apache.derby.impl.jdbc.EmbedSQLException: Java exception: ': java.lang.NullPointerException'. at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.SQLExceptionFactory40.wrapArgsForTransportAcrossDRDA(Unknown Source) [derby-10.8.1.2.jar:na] ... 27 common frames omitted Caused by: java.lang.NullPointerException: null at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.store.raw.data.BaseDataFileFactory.openContainer(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.store.raw.xact.Xact.openContainer(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.store.access.conglomerate.OpenConglomerate.init(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.store.access.heap.Heap.open(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.store.access.RAMTransaction.openConglomerate(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndexMinion(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getDescriptorViaIndex(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.locateSchemaRow(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.sql.catalog.DataDictionaryImpl.getSchemaDescriptor(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initDefaultSchemaDescriptor(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.sql.conn.GenericLanguageConnectionContext.initialize(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.db.BasicDatabase.setupConnection(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.TransactionResourceImpl.startTransaction(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.checkUserIsNotARole(Unknown Source) [derby-10.8.1.2.jar:na] at org.apache.derby.impl.jdbc.EmbedConnection.checkUserCredentials(Unknown Source) [derby-10.8.1.2.jar:na] ... 21 common frames omitted

            People

            • Assignee:
              Unassigned
              Reporter:
              vibhuti gupta
            • Votes:
              4 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:

                Development