Derby
  1. Derby
  2. DERBY-5343

Starting 7/13/2011 weme 6.2 upgrade tests started failing with 213 failures 127 errors with java.lang.IllegalAccessException

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.9.1.0
    • Fix Version/s: 10.9.1.0
    • Component/s: Test
    • Labels:
      None
    • Issue & fix info:
      High Value Fix
    • Bug behavior facts:
      Regression Test Failure

      Description

      Starting on 7/13/2011 weme 6.2 upgrade tests started failing with many failures and errors.
      The first Error is a java.lang.IllegalAccessException with stack trace below. It is probably related to this checkin.

      r1145973 | kahatlen | 2011-07-13 05:28:39 -0700 (Wed, 13 Jul 2011) | 3 lines

      DERBY-5316: Unload old JDBC drivers when done with them in the upgrade tests

      At first glance it looks like a JVM issue as the native method shoudln't get an IllegalAccessException.
      Probably best to work around it short term and disable the fix for weme. I think it just impacts the tests and not the product.

      Time: 11,927.828
      There were 127 errors:
      1) 10.1.1.0 Upgrade Phase: CREATE java.lang.IllegalAccessException
      at java.lang.reflect.Field.setImpl(Native Method)
      at java.lang.reflect.Field.set(Field.java:431)
      at org.apache.derbyTesting.functionTests.tests.upgradeTests.PhaseChanger.clearField(PhaseChanger.java:228)
      at org.apache.derbyTesting.functionTests.tests.upgradeTests.PhaseChanger.clearDerby23ThreadLocals(PhaseChanger.java:264)
      at org.apache.derbyTesting.functionTests.tests.upgradeTests.PhaseChanger.tearDown(PhaseChanger.java:153)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:20)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      2) testUnimplementedSQLObjectAttributes(org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest)SQL Exception: Feature not implemented: no details.
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.newEmbedSQLException(Unknown Source)
      at org.apache.derby.impl.jdbc.Util.notImplemented(Unknown Source)
      at org.apache.derby.impl.jdbc.EmbedDatabaseMetaData.getAttributes(Unknown Source)
      at org.apache.derbyTesting.functionTests.tests.jdbcapi.DatabaseMetaDataTest.testUnimplementedSQLObjectAttributes(DatabaseMetaDataTest.java:944)
      at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:195)
      at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)
      at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
      at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
      at junit.extensions.TestSetup.run(TestSetup.java:23)

      1. weme_failures.txt
        1.74 MB
        Kathey Marsden
      2. d5343.diff
        3 kB
        Knut Anders Hatlen

        Issue Links

          Activity

          Kathey Marsden created issue -
          Hide
          Kathey Marsden added a comment -

          attaching full test output. I think the first error is the real problem causing the others to cascade.

          Show
          Kathey Marsden added a comment - attaching full test output. I think the first error is the real problem causing the others to cascade.
          Kathey Marsden made changes -
          Field Original Value New Value
          Attachment weme_failures.txt [ 12486963 ]
          Hide
          Kathey Marsden added a comment -

          Actually the fields that we are trying to clear are private.
          In PhaseChanger we have:

          private void clearDerby23ThreadLocals(Object contextService)
          throws Exception {
          if (contextService != null)

          { Class cls = contextService.getClass(); clearField(cls, "threadContextList", contextService); clearField(cls, "allContexts", contextService); }

          }

          and in ContextService
          private HashSet allContexts;
          private ThreadLocal threadContextList = new ThreadLocal();

          With the other JVM's should we be able to clear these fields if they are private?

          Show
          Kathey Marsden added a comment - Actually the fields that we are trying to clear are private. In PhaseChanger we have: private void clearDerby23ThreadLocals(Object contextService) throws Exception { if (contextService != null) { Class cls = contextService.getClass(); clearField(cls, "threadContextList", contextService); clearField(cls, "allContexts", contextService); } } and in ContextService private HashSet allContexts; private ThreadLocal threadContextList = new ThreadLocal(); With the other JVM's should we be able to clear these fields if they are private?
          Hide
          Knut Anders Hatlen added a comment -

          I think clearing private fields should work in all JVMs, since that's what the call to f.setAccessible(true) is for. Those fields were final too before DERBY-23, and it looks like modifying final fields via reflection didn't work prior to Java 5, even when calling setAccessible() first. The tests also fail on 1.4.2 (both Sun and IBM).

          Show
          Knut Anders Hatlen added a comment - I think clearing private fields should work in all JVMs, since that's what the call to f.setAccessible(true) is for. Those fields were final too before DERBY-23 , and it looks like modifying final fields via reflection didn't work prior to Java 5, even when calling setAccessible() first. The tests also fail on 1.4.2 (both Sun and IBM).
          Knut Anders Hatlen made changes -
          Assignee Knut Anders Hatlen [ knutanders ]
          Knut Anders Hatlen made changes -
          Link This issue is broken by DERBY-5316 [ DERBY-5316 ]
          Knut Anders Hatlen made changes -
          Link This issue relates to DERBY-23 [ DERBY-23 ]
          Knut Anders Hatlen made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Hide
          Knut Anders Hatlen added a comment -

          Attaching a patch that makes the workaround for DERBY-23 take a different approach for making the engine classes eligible for garbage collection. Instead of setting the two final fields to null, it now does the following:

          • threadContextList: This field is a ThreadLocal, and the patch now calls ThreadLocal.set(null) on it to clear it in the current thread. We cannot clear it in other threads than the main JUnit thread this way, but assuming all other threads that have accessed the database are stopped and eventually garbage collected, this should be sufficient for the purpose of working around DERBY-23.
          • allContexts: This field is a HashSet, and it's now emptied by calling HashSet.clear().

          I've verified that the upgrade tests now pass on Java 1.4.2, and that the old engine classes are still getting garbage collected when they're no longer needed.

          Show
          Knut Anders Hatlen added a comment - Attaching a patch that makes the workaround for DERBY-23 take a different approach for making the engine classes eligible for garbage collection. Instead of setting the two final fields to null, it now does the following: threadContextList: This field is a ThreadLocal, and the patch now calls ThreadLocal.set(null) on it to clear it in the current thread. We cannot clear it in other threads than the main JUnit thread this way, but assuming all other threads that have accessed the database are stopped and eventually garbage collected, this should be sufficient for the purpose of working around DERBY-23 . allContexts: This field is a HashSet, and it's now emptied by calling HashSet.clear(). I've verified that the upgrade tests now pass on Java 1.4.2, and that the old engine classes are still getting garbage collected when they're no longer needed.
          Knut Anders Hatlen made changes -
          Attachment d5343.diff [ 12487001 ]
          Hide
          Knut Anders Hatlen added a comment -

          Committed revision 1148302.

          Show
          Knut Anders Hatlen added a comment - Committed revision 1148302.
          Knut Anders Hatlen made changes -
          Status In Progress [ 3 ] Resolved [ 5 ]
          Fix Version/s 10.9.0.0 [ 12316344 ]
          Resolution Fixed [ 1 ]
          Kathey Marsden made changes -
          Affects Version/s 10.9.0.0 [ 12316344 ]
          Hide
          Knut Anders Hatlen added a comment -

          [bulk update] Close all resolved issues that haven't been updated for more than one year.

          Show
          Knut Anders Hatlen added a comment - [bulk update] Close all resolved issues that haven't been updated for more than one year.
          Knut Anders Hatlen made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Gavin made changes -
          Workflow jira [ 12620846 ] Default workflow, editable Closed status [ 12802870 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open In Progress In Progress
          9h 57m 1 Knut Anders Hatlen 19/Jul/11 13:25
          In Progress In Progress Resolved Resolved
          25m 18s 1 Knut Anders Hatlen 19/Jul/11 13:50
          Resolved Resolved Closed Closed
          698d 20h 29m 1 Knut Anders Hatlen 17/Jun/13 10:19

            People

            • Assignee:
              Knut Anders Hatlen
              Reporter:
              Kathey Marsden
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development