Derby
  1. Derby
  2. DERBY-5081

Intermittent failure in InterruptResilienceTest: IO thread times out waiting for another thread to recover channel

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Invalid
    • Affects Version/s: 10.8.1.2
    • Fix Version/s: None
    • Component/s: Store, Test
    • Labels:
      None
    • Environment:
      Linux/JDK 7
      Java 7 (b135) on Debian 5.0
      OEL/JDK7
    • Bug behavior facts:
      Regression Test Failure

      Description

      A couple of failures have been seen in InterruptResilienceTest lately: (Edit: Dag. this issue is for the first one only, the second issue has been allocated DERBY-5140)

      http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/lin/1075421-suitesAll_diff.txt :

      testRAFWriteInterrupted(org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest)java.sql.SQLException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 38000, SQLERRMC: java.sql.SQLException: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt.38000XSDG9:XSDG9.D
      at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source)
      at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source)
      at org.apache.derby.client.am.Statement.executeUpdate(Unknown Source)
      at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest.testRAFWriteInterrupted(InterruptResilienceTest.java:204)

      [edit: treatment of this is moved to DERBY-5140: Windows 2003/Java 1.4.2 - http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.4/testing/testlog/w2003/1075087-suitesAll_diff.txt:

      testRAFReadWriteMultipleThreads(org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest)java.sql.SQLException: The exception 'java.lang.InterruptedException' was thrown while evaluating an expression.
      at org.apache.derby.impl.jdbc.SQLExceptionFactory.getSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.seeNextException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.wrapInSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.TransactionResourceImpl.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedConnection.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.ConnectionChild.handleException(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.executeStatement(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.execute(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedStatement.executeUpdate(Unknown Source)
      at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest.testRAFReadWriteMultipleThreads(InterruptResilienceTest.java:515)
      ]

      1. derby.log
        157 kB
        Knut Anders Hatlen
      2. derby.log
        151 kB
        Knut Anders Hatlen
      3. derby-5081-compensate.diff
        1 kB
        Dag H. Wanvik

        Issue Links

          Activity

          Hide
          Dag H. Wanvik added a comment -

          These are probably two unrelated errors. The first one seems a RAF recovery gone awry, it would be nice to see if we can locate derby.log for it (the link is client stack trace). It is interesting that this is JDK7 because the interrupt treatment in NIO has some bug fixes there..

          Show
          Dag H. Wanvik added a comment - These are probably two unrelated errors. The first one seems a RAF recovery gone awry, it would be nice to see if we can locate derby.log for it (the link is client stack trace). It is interesting that this is JDK7 because the interrupt treatment in NIO has some bug fixes there..
          Hide
          Knut Anders Hatlen added a comment -

          The JDK 7 tests still use a somewhat old build (b116) that probably doesn't contain those bug fixes. At least the bug http://bugs.sun.com/view_bug.do?bug_id=6979009 mentioned in DERBY-4741 wasn't fixed until b120.

          Show
          Knut Anders Hatlen added a comment - The JDK 7 tests still use a somewhat old build (b116) that probably doesn't contain those bug fixes. At least the bug http://bugs.sun.com/view_bug.do?bug_id=6979009 mentioned in DERBY-4741 wasn't fixed until b120.
          Hide
          Knut Anders Hatlen added a comment -

          Attached derby.log from the failing test on JDK 7.

          Show
          Knut Anders Hatlen added a comment - Attached derby.log from the failing test on JDK 7.
          Hide
          Rick Hillegas added a comment -

          Has the Linux/JDK7 variant of this bug surfaced since upgrading to a later build of JDK 7? Thanks.

          Show
          Rick Hillegas added a comment - Has the Linux/JDK7 variant of this bug surfaced since upgrading to a later build of JDK 7? Thanks.
          Hide
          Knut Anders Hatlen added a comment -

          I haven't seen it after the upgrade of JDK 7. There have been other failures in InterruptResilienceTest after the upgrade because of DERBY-5104 (fixed now), which may have masked this problem.

          Show
          Knut Anders Hatlen added a comment - I haven't seen it after the upgrade of JDK 7. There have been other failures in InterruptResilienceTest after the upgrade because of DERBY-5104 (fixed now), which may have masked this problem.
          Hide
          Dag H. Wanvik added a comment - - edited

          Issue 1) Thanks for the upload of derby.log. Knut. A thread is waiting for recovery assumed to be performed by another thread, which never (at least within a minute never happens). I can't tell why from the log. In any case, this behavior shuts down the database, and is thus no worse than the current Derby behavior. If we see this again with some frequency, I may try to instrument to code on trunk so we can get an insight to why this is sometimes happening.

          Issue 2) DERBY-5140 This part of the stack is interesting:

          Caused by: java.lang.InterruptedException
          at java.lang.Object.wait(Native Method)
          at java.lang.Thread.join(Thread.java:1001)
          at java.lang.Thread.join(Thread.java:1054)
          at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest.tstRAFReadWriteMultipleThreads(InterruptResilienceTest.java:244)
          at org.apache.derby.exe.ac070a00b0x012ex6c08x9759x00006a12110b0.g0(Unknown Source)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source)
          at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source)
          at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source)

          It shows that the thread driving the test sees an interrupt (in the
          join waiting for the worked threads to terminate. Now, the interruptor
          threads should only be interrupting the worker threads themselves, not
          the driver test thread that started (and is now waiting for) the
          worker threads. I am not sure how this join could get interrupted,
          because the specification of Thread#join states that it will only
          throw InterruptedException iff the thread doing the join (i.e. not if the
          joinee) is interrupted.

          Possible theories:
          a) That somehow the exception is delivered to the wrong thread iff the
          worker thread is in the final stages when it receives an interrupt
          from the interruptor threads

          b) The interrupts emanates from Derby shutdown, which his hard to see
          could be happening here: we are still inside a test
          (testRAFReadWriteMultipleThreads), in embedded mode.

          Note that this run has already seen a failure in
          AutomaticIndexStatisticsTest#testShutdownWhileScanningThenDelete prior
          to hitting the error in InterruptResilienceTest. I don't have no
          reason to suspect that the two are related at this point, though.

          I'll split this bug in two.

          Show
          Dag H. Wanvik added a comment - - edited Issue 1) Thanks for the upload of derby.log. Knut. A thread is waiting for recovery assumed to be performed by another thread, which never (at least within a minute never happens). I can't tell why from the log. In any case, this behavior shuts down the database, and is thus no worse than the current Derby behavior. If we see this again with some frequency, I may try to instrument to code on trunk so we can get an insight to why this is sometimes happening. Issue 2) DERBY-5140 This part of the stack is interesting: Caused by: java.lang.InterruptedException at java.lang.Object.wait(Native Method) at java.lang.Thread.join(Thread.java:1001) at java.lang.Thread.join(Thread.java:1054) at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest.tstRAFReadWriteMultipleThreads(InterruptResilienceTest.java:244) at org.apache.derby.exe.ac070a00b0x012ex6c08x9759x00006a12110b0.g0(Unknown Source) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.apache.derby.impl.services.reflect.ReflectMethod.invoke(Unknown Source) at org.apache.derby.impl.sql.execute.CallStatementResultSet.open(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.executeStmt(Unknown Source) at org.apache.derby.impl.sql.GenericPreparedStatement.execute(Unknown Source) It shows that the thread driving the test sees an interrupt (in the join waiting for the worked threads to terminate. Now, the interruptor threads should only be interrupting the worker threads themselves, not the driver test thread that started (and is now waiting for) the worker threads. I am not sure how this join could get interrupted, because the specification of Thread#join states that it will only throw InterruptedException iff the thread doing the join (i.e. not if the joinee) is interrupted. Possible theories: a) That somehow the exception is delivered to the wrong thread iff the worker thread is in the final stages when it receives an interrupt from the interruptor threads b) The interrupts emanates from Derby shutdown, which his hard to see could be happening here: we are still inside a test (testRAFReadWriteMultipleThreads), in embedded mode. Note that this run has already seen a failure in AutomaticIndexStatisticsTest#testShutdownWhileScanningThenDelete prior to hitting the error in InterruptResilienceTest. I don't have no reason to suspect that the two are related at this point, though. I'll split this bug in two.
          Hide
          Knut Anders Hatlen added a comment -

          I saw a similar stack trace again. This time on a newer Java 7 (b135) on Debian 5.0. Here's the stack trace reported by JUnit:

          java.sql.SQLException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 38000, SQLERRMC: java.sql.SQLException: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt.38000XSDG9:XSDG9.D
          at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source)
          at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source)
          at org.apache.derby.client.am.Statement.executeUpdate(Unknown Source)
          at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest.testRAFWriteInterrupted(InterruptResilienceTest.java:217)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 38000, SQLERRMC: java.sql.SQLException: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt.38000XSDG9:XSDG9.D
          at org.apache.derby.client.am.Statement.completeExecute(Unknown Source)
          at org.apache.derby.client.net.NetStatementReply.parseEXCSQLSTTreply(Unknown Source)
          at org.apache.derby.client.net.NetStatementReply.readExecuteCall(Unknown Source)
          at org.apache.derby.client.net.StatementReply.readExecuteCall(Unknown Source)
          at org.apache.derby.client.net.NetStatement.readExecuteCall_(Unknown Source)
          at org.apache.derby.client.am.Statement.readExecuteCall(Unknown Source)
          at org.apache.derby.client.am.Statement.flowExecute(Unknown Source)
          at org.apache.derby.client.am.Statement.executeUpdateX(Unknown Source)
          Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: 0, SQLSTATE: XSDG9, SQLERRMC: XSDG9.D

          Show
          Knut Anders Hatlen added a comment - I saw a similar stack trace again. This time on a newer Java 7 (b135) on Debian 5.0. Here's the stack trace reported by JUnit: java.sql.SQLException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 38000, SQLERRMC: java.sql.SQLException: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt.38000XSDG9:XSDG9.D at org.apache.derby.client.am.SQLExceptionFactory40.getSQLException(Unknown Source) at org.apache.derby.client.am.SqlException.getSQLException(Unknown Source) at org.apache.derby.client.am.Statement.executeUpdate(Unknown Source) at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest.testRAFWriteInterrupted(InterruptResilienceTest.java:217) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: -1, SQLSTATE: 38000, SQLERRMC: java.sql.SQLException: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt.38000XSDG9:XSDG9.D at org.apache.derby.client.am.Statement.completeExecute(Unknown Source) at org.apache.derby.client.net.NetStatementReply.parseEXCSQLSTTreply(Unknown Source) at org.apache.derby.client.net.NetStatementReply.readExecuteCall(Unknown Source) at org.apache.derby.client.net.StatementReply.readExecuteCall(Unknown Source) at org.apache.derby.client.net.NetStatement.readExecuteCall_(Unknown Source) at org.apache.derby.client.am.Statement.readExecuteCall(Unknown Source) at org.apache.derby.client.am.Statement.flowExecute(Unknown Source) at org.apache.derby.client.am.Statement.executeUpdateX(Unknown Source) Caused by: org.apache.derby.client.am.SqlException: DERBY SQL error: SQLCODE: 0, SQLSTATE: XSDG9, SQLERRMC: XSDG9.D
          Hide
          Knut Anders Hatlen added a comment -

          Uploading derby.log for the latest failure. Not sure if it's identical to the originally reported problem, but it looks like both are caused by an exception in the background cleaner:

          ERROR XSDG9: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt.
          at org.apache.derby.iapi.error.StandardException.newException(Unknown Source)
          at org.apache.derby.impl.store.raw.data.RAFContainer4.writePage(Unknown Source)
          at org.apache.derby.impl.store.raw.data.CachedPage.writePage(Unknown Source)
          at org.apache.derby.impl.store.raw.data.CachedPage.clean(Unknown Source)
          at org.apache.derby.impl.services.cache.ConcurrentCache.cleanAndUnkeepEntry(Unknown Source)
          at org.apache.derby.impl.services.cache.ConcurrentCache.cleanEntry(Unknown Source)
          at org.apache.derby.impl.services.cache.BackgroundCleaner.performWork(Unknown Source)
          at org.apache.derby.impl.services.daemon.BasicDaemon.serviceClient(Unknown Source)
          at org.apache.derby.impl.services.daemon.BasicDaemon.work(Unknown Source)
          at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source)
          at java.lang.Thread.run(Thread.java:722)

          Show
          Knut Anders Hatlen added a comment - Uploading derby.log for the latest failure. Not sure if it's identical to the originally reported problem, but it looks like both are caused by an exception in the background cleaner: ERROR XSDG9: Derby thread received an interrupt during a disk I/O operation, please check your application for the source of the interrupt. at org.apache.derby.iapi.error.StandardException.newException(Unknown Source) at org.apache.derby.impl.store.raw.data.RAFContainer4.writePage(Unknown Source) at org.apache.derby.impl.store.raw.data.CachedPage.writePage(Unknown Source) at org.apache.derby.impl.store.raw.data.CachedPage.clean(Unknown Source) at org.apache.derby.impl.services.cache.ConcurrentCache.cleanAndUnkeepEntry(Unknown Source) at org.apache.derby.impl.services.cache.ConcurrentCache.cleanEntry(Unknown Source) at org.apache.derby.impl.services.cache.BackgroundCleaner.performWork(Unknown Source) at org.apache.derby.impl.services.daemon.BasicDaemon.serviceClient(Unknown Source) at org.apache.derby.impl.services.daemon.BasicDaemon.work(Unknown Source) at org.apache.derby.impl.services.daemon.BasicDaemon.run(Unknown Source) at java.lang.Thread.run(Thread.java:722)
          Hide
          Dag H. Wanvik added a comment -

          Right, I'll try to go after this one then. It seems to be more likely (at least) on a Linux/JDK7 platform, so I'll start some overnight runs to see if I can reproduce this with some frequency. Thanks.

          Show
          Dag H. Wanvik added a comment - Right, I'll try to go after this one then. It seems to be more likely (at least) on a Linux/JDK7 platform, so I'll start some overnight runs to see if I can reproduce this with some frequency. Thanks.
          Hide
          Dag H. Wanvik added a comment - - edited

          I have been able to reproduce this on Linux with JDK7. Instrumentation shows that this is a JDK issue: The API guarantee is not beeing honored, cf. the following (instrumented to catch the situation) code from RAFContainer4:

          private void writeFull(ByteBuffer srcBuffer,
          FileChannel dstChannel,
          long position)
          throws IOException
          {
          try {
          while(srcBuffer.remaining() > 0) {
          dstChannel.write(srcBuffer, position + srcBuffer.position());

          // (**) Sun JAVA NIO is weird: it can close the channel due to an
          // interrupt without throwing if bytes got transferred. Compensate,
          // so we can clean up. Bug 6979009,
          // http://bugs.sun.com/view_bug.do?bug_id=6979009
          if (Thread.currentThread().isInterrupted() &&
          !dstChannel.isOpen())

          { throw new ClosedByInterruptException(); }

          }
          } catch (ClosedByInterruptException e) {
          if (!Thread.currentThread().isInterrupted()) {

          // while (true) {
          ---> // try

          { Thread.sleep(200000); }

          catch (Exception ee) {}
          // }

          // Compensate for JDK 7 on Linux (b141):
          Thread.currentThread().interrupt();
          }
          throw e;
          }
          }

          That is, when the thread is interrupted and the channel write throws
          ClosedByInterruptException, the interrupt flag is not set, as it
          should be as per the API, cf this sentence in the Javadoc for
          ClosedByInterruptException:

          "Checked exception received by a thread when another thread interrupts
          it while it is blocked in an I/O operation upon a channel. Before this
          exception is thrown the channel will have been closed and the
          interrupt status of the previously-blocked thread will have been set."

          Note that in our test case, the thread's is interrupted before
          the FileChannel#write is attempted. I believe that the API contract
          should still hold, though.

          With this fix (setting the flag as shown above), I can't reproduces the issue in 100 runs, where as without it I see it in every 5-10 runs or so.

          Show
          Dag H. Wanvik added a comment - - edited I have been able to reproduce this on Linux with JDK7. Instrumentation shows that this is a JDK issue: The API guarantee is not beeing honored, cf. the following (instrumented to catch the situation) code from RAFContainer4: private void writeFull(ByteBuffer srcBuffer, FileChannel dstChannel, long position) throws IOException { try { while(srcBuffer.remaining() > 0) { dstChannel.write(srcBuffer, position + srcBuffer.position()); // (**) Sun JAVA NIO is weird: it can close the channel due to an // interrupt without throwing if bytes got transferred. Compensate, // so we can clean up. Bug 6979009, // http://bugs.sun.com/view_bug.do?bug_id=6979009 if (Thread.currentThread().isInterrupted() && !dstChannel.isOpen()) { throw new ClosedByInterruptException(); } } } catch (ClosedByInterruptException e) { if (!Thread.currentThread().isInterrupted()) { // while (true) { ---> // try { Thread.sleep(200000); } catch (Exception ee) {} // } // Compensate for JDK 7 on Linux (b141): Thread.currentThread().interrupt(); } throw e; } } That is, when the thread is interrupted and the channel write throws ClosedByInterruptException, the interrupt flag is not set, as it should be as per the API, cf this sentence in the Javadoc for ClosedByInterruptException: "Checked exception received by a thread when another thread interrupts it while it is blocked in an I/O operation upon a channel. Before this exception is thrown the channel will have been closed and the interrupt status of the previously-blocked thread will have been set." Note that in our test case, the thread's is interrupted before the FileChannel#write is attempted. I believe that the API contract should still hold, though. With this fix (setting the flag as shown above), I can't reproduces the issue in 100 runs, where as without it I see it in every 5-10 runs or so.
          Hide
          Dag H. Wanvik added a comment -

          Uploading a patch, derby-5081-compensate, which could be used to avoid this issue. If this issue is fixed in the upcoming release of Sun JDK 7, I would avoid committing it. I have reported this issue to the JDK folks; depending on the outcome I will suggest a course of action on this patch: I think if the issue is fixed in the first official JDK 7 release , it would be nice to avoid committing this kludge.

          Show
          Dag H. Wanvik added a comment - Uploading a patch, derby-5081-compensate, which could be used to avoid this issue. If this issue is fixed in the upcoming release of Sun JDK 7, I would avoid committing it. I have reported this issue to the JDK folks; depending on the outcome I will suggest a course of action on this patch: I think if the issue is fixed in the first official JDK 7 release , it would be nice to avoid committing this kludge.
          Hide
          Dag H. Wanvik added a comment - - edited

          Also on this platform combo, I sometimes see another error which I suspect is related:

          ..Exception in thread "WorkerThread. Thread#5" junit.framework.AssertionFailedError
          at junit.framework.Assert.fail(Assert.java:47)
          at junit.framework.Assert.assertTrue(Assert.java:20)
          at junit.framework.Assert.assertTrue(Assert.java:27)
          at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest$WorkerThread.run(InterruptResilienceTest.java:430)

          In this case also, the reason is that a tread's interrupted flag was lost (during re-preparing a statement). It could be the same underlying error. If, when we get a fix for the JDK 7, it doesn't remove this symptom, we should file a different issue for it.

          [edit: I filed DERBY-5223 for this - it is a different issue]

          Show
          Dag H. Wanvik added a comment - - edited Also on this platform combo, I sometimes see another error which I suspect is related: ..Exception in thread "WorkerThread. Thread#5" junit.framework.AssertionFailedError at junit.framework.Assert.fail(Assert.java:47) at junit.framework.Assert.assertTrue(Assert.java:20) at junit.framework.Assert.assertTrue(Assert.java:27) at org.apache.derbyTesting.functionTests.tests.store.InterruptResilienceTest$WorkerThread.run(InterruptResilienceTest.java:430) In this case also, the reason is that a tread's interrupted flag was lost (during re-preparing a statement). It could be the same underlying error. If, when we get a fix for the JDK 7, it doesn't remove this symptom, we should file a different issue for it. [edit: I filed DERBY-5223 for this - it is a different issue]
          Hide
          Knut Anders Hatlen added a comment -

          Hi Dag,

          The stack trace for the AssertionFailedError looks like something that's printed to the console. Did the test also fail because of it? If not, do you think it's possible to recode the test to detect failures in the worker threads?

          By the way, similar stack traces in WorkerThread have been seen printed to the console on Windows too, Java 1.4.2 and Java 6:

          http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.4/testing/testlog/w2003/1078053-suitesAll_diff.txt
          http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.4/testing/testlog/w2003/1080669-suitesAll_diff.txt
          http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.6/testing/testlog/w2003/1084638-suitesAll_diff.txt

          (Only in one of these runs InterruptResilienceTest actually failed.)

          Show
          Knut Anders Hatlen added a comment - Hi Dag, The stack trace for the AssertionFailedError looks like something that's printed to the console. Did the test also fail because of it? If not, do you think it's possible to recode the test to detect failures in the worker threads? By the way, similar stack traces in WorkerThread have been seen printed to the console on Windows too, Java 1.4.2 and Java 6: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.4/testing/testlog/w2003/1078053-suitesAll_diff.txt http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.4/testing/testlog/w2003/1080669-suitesAll_diff.txt http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.6/testing/testlog/w2003/1084638-suitesAll_diff.txt (Only in one of these runs InterruptResilienceTest actually failed.)
          Hide
          Dag H. Wanvik added a comment - - edited

          [edit: I filed DERBY-5223 for this - it is a different issue]

          Thanks Knut. You are right, the test did not fail (JUnit assert in thread doesn't work as one could assume). I'll change that. Since a similar "loss of interrupt flag" has also happened on Windows, this is probably not the same error as this issue. I'll investigate it.

          Show
          Dag H. Wanvik added a comment - - edited [edit: I filed DERBY-5223 for this - it is a different issue] Thanks Knut. You are right, the test did not fail (JUnit assert in thread doesn't work as one could assume). I'll change that. Since a similar "loss of interrupt flag" has also happened on Windows, this is probably not the same error as this issue. I'll investigate it.
          Hide
          Dag H. Wanvik added a comment -

          It seems this issue will be fixed in the JDK, so i'll hold off committing any patch (derby-5081-compensate is just an ugly work-around really) for this issue until we know if the fix makes it into the official JDK 7.

          Show
          Dag H. Wanvik added a comment - It seems this issue will be fixed in the JDK, so i'll hold off committing any patch (derby-5081-compensate is just an ugly work-around really) for this issue until we know if the fix makes it into the official JDK 7.
          Hide
          Knut Anders Hatlen added a comment -

          The JDK bug that caused this was http://bugs.sun.com/view_bug.do?bug_id=7043425. It's fixed in JDK 7 b143.

          Show
          Knut Anders Hatlen added a comment - The JDK bug that caused this was http://bugs.sun.com/view_bug.do?bug_id=7043425 . It's fixed in JDK 7 b143.
          Hide
          Myrna van Lunteren added a comment -

          Can this issue be closed now?

          Show
          Myrna van Lunteren added a comment - Can this issue be closed now?
          Hide
          Knut Anders Hatlen added a comment -

          Closing as invalid since it was a JVM bug.

          Show
          Knut Anders Hatlen added a comment - Closing as invalid since it was a JVM bug.

            People

            • Assignee:
              Unassigned
              Reporter:
              Knut Anders Hatlen
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development