Derby
  1. Derby
  2. DERBY-4179

bootLock.java fails with missing exception on z/OS with pmz3160sr2ifix-20081021_01(SR2+IZ32776+IZ33456), and Windows Vista

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.5.1.1
    • Fix Version/s: 10.5.3.1, 10.6.2.1, 10.7.1.1
    • Component/s: Store, Test
    • Labels:
      None
    • Environment:
    • Issue & fix info:
      Patch Available

      Description

      I saw this diff in store/bootLock.java. I did not see it with the 64bit jvm run on 10.5.1.0 RC1

          • Start: bootLock jdk1.6.0 storeall:storemore 2009-04-21 19:10:18 ***
            2,4d1
            < expected exception
            < SQLSTATE(XJ040):
            < SQLSTATE(XSDB6):
            Test Failed.
          • End: bootLock jdk1.6.0 storeall:storemore 2009-04-21 19:11:00 ***

      The test passed on rerun when run independently.

      1. derby-4179-junit-5.stat
        1.0 kB
        Dag H. Wanvik
      2. derby-4179-junit-5.diff
        28 kB
        Dag H. Wanvik
      3. derby-4179-junit-4.stat
        1.0 kB
        Dag H. Wanvik
      4. derby-4179-junit-4.diff
        27 kB
        Dag H. Wanvik
      5. derby-4179-junit-3.stat
        0.8 kB
        Dag H. Wanvik
      6. derby-4179-junit-3.diff
        20 kB
        Dag H. Wanvik
      7. derby-4179-junit-2.stat
        0.8 kB
        Dag H. Wanvik
      8. derby-4179-junit-2.diff
        19 kB
        Dag H. Wanvik
      9. derby-4179-junit.stat
        0.8 kB
        Dag H. Wanvik
      10. derby-4179-junit.diff
        19 kB
        Dag H. Wanvik
      11. derby-4179.stat
        0.2 kB
        Dag H. Wanvik
      12. derby-4179.diff
        5 kB
        Dag H. Wanvik
      13. sysinfo.txt
        3 kB
        Dag H. Wanvik
      14. bootLock.diff
        2 kB
        Sylvain Leroux
      15. storeall_report.txt
        3 kB
        Sylvain Leroux
      16. derbyall_report.txt
        4 kB
        Sylvain Leroux

        Issue Links

          Activity

          Hide
          Kathey Marsden added a comment -

          This happened just once and with an older version of the JVM, so I think we won't fix it unless it comes up again.

          Show
          Kathey Marsden added a comment - This happened just once and with an older version of the JVM, so I think we won't fix it unless it comes up again.
          Hide
          Knut Anders Hatlen added a comment -

          This has happened again on a different JVM, so I'm reopening the issue. See comment by Sylvain Leroux on DERBY-4430, 14/Nov/09 12:18 PM.

          Show
          Knut Anders Hatlen added a comment - This has happened again on a different JVM, so I'm reopening the issue. See comment by Sylvain Leroux on DERBY-4430 , 14/Nov/09 12:18 PM.
          Hide
          Sylvain Leroux added a comment -

          As a reference, I attach the corresponding *_report.txt. I'm running on Debien/Lenny with a custom kernel.

          The problem appears to be reproducible on my specific environment.

          Show
          Sylvain Leroux added a comment - As a reference, I attach the corresponding *_report.txt. I'm running on Debien/Lenny with a custom kernel. The problem appears to be reproducible on my specific environment.
          Hide
          Dag H. Wanvik added a comment - - edited

          I also saw this now on the db-derby-10.6.1.0 RC1 (insane) on Windows Vista (32 bits), Sun JDK 1.6.0_07-b06 running under Cygwin. It is reproducable
          in my enviroment also when run alone (sane and insane jars).

          Show
          Dag H. Wanvik added a comment - - edited I also saw this now on the db-derby-10.6.1.0 RC1 (insane) on Windows Vista (32 bits), Sun JDK 1.6.0_07-b06 running under Cygwin. It is reproducable in my enviroment also when run alone (sane and insane jars).
          Hide
          Dag H. Wanvik added a comment -

          I ran the test on trunk (svn 939730) with both classes and sane jars, otherwise same environment (I think!), but could not reproduce the error there. Hmm.

          Show
          Dag H. Wanvik added a comment - I ran the test on trunk (svn 939730) with both classes and sane jars, otherwise same environment (I think!), but could not reproduce the error there. Hmm.
          Hide
          Dag H. Wanvik added a comment -

          Attaching sysinfo.txt for the case which fails. I tried to use the same classpath but with absolute jar paths, but then I didn't see the error. That could be a red herring of course; I'll try to narrow it down a bit.

          Show
          Dag H. Wanvik added a comment - Attaching sysinfo.txt for the case which fails. I tried to use the same classpath but with absolute jar paths, but then I didn't see the error. That could be a red herring of course; I'll try to narrow it down a bit.
          Hide
          Dag H. Wanvik added a comment -

          Using this (relative) minimal classpath I was able to reproduce on trunk/cygwin;

          >echo $CLASSPATH
          ./jars/sane/derby.jar;./jars/sane/derbytools.jar;./jars/sane/derbyTesting.jar;./tools/java/jakarta-oro-2.0.8.jar;./tools/java/junit.jar

          >java org.apache.derbyTesting.functionTests.harness.RunTest store/bootLock.java

              • Start: bootLock jdk1.6.0_07 2010-05-02 21:13:49 ***
            • SecurityManager not installed –
              2,4d1
              < expected exception
              < SQLSTATE(XJ040):
              < SQLSTATE(XSDB6):
              Test Failed.

          Making the CLASSPATH consist of absolute jar references makes the problem go away.

          Show
          Dag H. Wanvik added a comment - Using this (relative) minimal classpath I was able to reproduce on trunk/cygwin; >echo $CLASSPATH ./jars/sane/derby.jar;./jars/sane/derbytools.jar;./jars/sane/derbyTesting.jar;./tools/java/jakarta-oro-2.0.8.jar;./tools/java/junit.jar >java org.apache.derbyTesting.functionTests.harness.RunTest store/bootLock.java Start: bootLock jdk1.6.0_07 2010-05-02 21:13:49 *** SecurityManager not installed – 2,4d1 < expected exception < SQLSTATE(XJ040): < SQLSTATE(XSDB6): Test Failed. Making the CLASSPATH consist of absolute jar references makes the problem go away.
          Hide
          Dag H. Wanvik added a comment -

          Testing relative jar paths on trunk / Solaris, I see the same symptom. Running test with useprocess=false made no difference.

          Show
          Dag H. Wanvik added a comment - Testing relative jar paths on trunk / Solaris, I see the same symptom. Running test with useprocess=false made no difference.
          Hide
          Kathey Marsden added a comment -

          Thank you Dag and Sylvain for tracking this one down. Any dual boot opportunity is very very bad and hard to track down in a production environment. especially if sensitive to subtle classpath changes. I kind of doubt it is the same thing I saw when I originally logged this issue as I typically set up scripts for testing with absolute paths and would have used the same script when I upgraded the JVM that made the problem go away. Does the relative path issue reproduce on 10.5 or is it a 10.6 regression? Especially if it is a regression and maybe if it is not, it might make sense to file a new bug.

          Show
          Kathey Marsden added a comment - Thank you Dag and Sylvain for tracking this one down. Any dual boot opportunity is very very bad and hard to track down in a production environment. especially if sensitive to subtle classpath changes. I kind of doubt it is the same thing I saw when I originally logged this issue as I typically set up scripts for testing with absolute paths and would have used the same script when I upgraded the JVM that made the problem go away. Does the relative path issue reproduce on 10.5 or is it a 10.6 regression? Especially if it is a regression and maybe if it is not, it might make sense to file a new bug.
          Hide
          Dag H. Wanvik added a comment -

          bootLock.java starts another VM; this may be a lead.

          Show
          Dag H. Wanvik added a comment - bootLock.java starts another VM; this may be a lead.
          Hide
          Dag H. Wanvik added a comment -

          Ok, the test framework starts bootLock in a subdirectory (different from the cwd when the test is started), so the forked
          process (bootLock1), will have the wrong CLASSPATH: since it consists of relative paths to the jars, i.e. relative to the directory in which the test is started, not relative to the created subdirectory, the forked subprocess will fail, leaving the way open for bootLock to open wombat without seeing it already opened/locked. No dual boot is really happening

          As far as I can see, this is a test bug, not a product bug, at least in the incarnation I am seeing.

          Show
          Dag H. Wanvik added a comment - Ok, the test framework starts bootLock in a subdirectory (different from the cwd when the test is started), so the forked process (bootLock1), will have the wrong CLASSPATH: since it consists of relative paths to the jars, i.e. relative to the directory in which the test is started , not relative to the created subdirectory, the forked subprocess will fail, leaving the way open for bootLock to open wombat without seeing it already opened/locked. No dual boot is really happening As far as I can see, this is a test bug, not a product bug, at least in the incarnation I am seeing.
          Hide
          Dag H. Wanvik added a comment -

          Kathey, If you think this incarnation has a different root cause than yours, I'll file a new issue for it.

          Show
          Dag H. Wanvik added a comment - Kathey, If you think this incarnation has a different root cause than yours, I'll file a new issue for it.
          Hide
          Kathey Marsden added a comment -

          Dag said:
          >No dual boot is really happening.

          Thank goodness! I really didn't sleep all that well thinking this was back. I guess then it would be fine to leave it on this bug since this is where all the comments are and you understand the problem now and it is also on 10.5, The history is here to show the original thing might have been something else, if we happen to ever see it resurface on z/os and if we do can file another bug for that one.

          Show
          Kathey Marsden added a comment - Dag said: >No dual boot is really happening. Thank goodness! I really didn't sleep all that well thinking this was back. I guess then it would be fine to leave it on this bug since this is where all the comments are and you understand the problem now and it is also on 10.5, The history is here to show the original thing might have been something else, if we happen to ever see it resurface on z/os and if we do can file another bug for that one.
          Hide
          Dag H. Wanvik added a comment -

          Adding test component as well.

          Show
          Dag H. Wanvik added a comment - Adding test component as well.
          Hide
          Dag H. Wanvik added a comment -

          Here is a patch which makes store/bootLock.java work in the presence of a classpath with relative pathnames. I tested manually on Solaris and Windows/cygwin, with variants of CLASSPATH env vars. I addmittedly presumed any explicit -cp args gets picked up by system property java.class.path as well..

          A weakness is that I had to up the source level to 1.5 to be able to read the process environment to be able to replace
          just the value of the system property java.class.path in order to make a suitable new enviroment to pass on to the forkee. The approach feels like a bit of a hack; better suggestions are welcome!

          I am not sure if we require that tests be runnable with 1.4, or just the product.. Running full regressions.

          Please review.

          Show
          Dag H. Wanvik added a comment - Here is a patch which makes store/bootLock.java work in the presence of a classpath with relative pathnames. I tested manually on Solaris and Windows/cygwin, with variants of CLASSPATH env vars. I addmittedly presumed any explicit -cp args gets picked up by system property java.class.path as well.. A weakness is that I had to up the source level to 1.5 to be able to read the process environment to be able to replace just the value of the system property java.class.path in order to make a suitable new enviroment to pass on to the forkee. The approach feels like a bit of a hack; better suggestions are welcome! I am not sure if we require that tests be runnable with 1.4, or just the product.. Running full regressions. Please review.
          Hide
          Kathey Marsden added a comment -

          Would perhaps converting the test to JUnit provide cleaner options? BaseTestCase.execJavaCmd() uses java.class.path as is, but perhaps you would not have the problem with the nested directory.

          Show
          Kathey Marsden added a comment - Would perhaps converting the test to JUnit provide cleaner options? BaseTestCase.execJavaCmd() uses java.class.path as is, but perhaps you would not have the problem with the nested directory.
          Hide
          Dag H. Wanvik added a comment -

          Thanks Kathey, yes, I think that's probably a better way out here... I started with just a tiny fix, but it grew and by the time I had it working I didn't like it... I'll have a go at a rewrite.

          Show
          Dag H. Wanvik added a comment - Thanks Kathey, yes, I think that's probably a better way out here... I started with just a tiny fix, but it grew and by the time I had it working I didn't like it... I'll have a go at a rewrite.
          Hide
          Knut Anders Hatlen added a comment -

          I think it would also be useful to make the test report that the sub-process failed. If the original test had just printed a line saying "sub-process terminated with exit code 1" or something, we would probably have known earlier where to look.

          Show
          Knut Anders Hatlen added a comment - I think it would also be useful to make the test report that the sub-process failed. If the original test had just printed a line saying "sub-process terminated with exit code 1" or something, we would probably have known earlier where to look.
          Hide
          Dag H. Wanvik added a comment -

          Uploading a patch which rewrites store/bootLock to JUnit. If the test fails, the subprocess will leave its log file around for inspection. It appears to run happily with relative jars.

          I notice that the old test properties had these settings:
          -runwithj9=false
          -runwithibm13=false
          -runwithjdk13=false

          The latter is stale and probably ibm13 as well, but please let me know if the "j9" exception is still relevant, and if so, how my test should skip the test for this platform.

          Show
          Dag H. Wanvik added a comment - Uploading a patch which rewrites store/bootLock to JUnit. If the test fails, the subprocess will leave its log file around for inspection. It appears to run happily with relative jars. I notice that the old test properties had these settings: -runwithj9=false -runwithibm13=false -runwithjdk13=false The latter is stale and probably ibm13 as well , but please let me know if the "j9" exception is still relevant, and if so, how my test should skip the test for this platform.
          Hide
          Kathey Marsden added a comment -

          I am guessing the old runwithj9=false because the spawned process stuff did not work with the old harness and j9, but I just tried the new test with weme6.1 and it failed with:
          1) testBootLock(org.apache.derbyTesting.functionTests.tests.store.BootLockTest)j
          unit.framework.AssertionFailedError: Dual boot not detected: check BootLockMinio
          n.log
          at org.apache.derbyTesting.functionTests.tests.store.BootLockTest.testBo
          otLock(BootLockTest.java:92)
          at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:205)

          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:
          109)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57
          )
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:19)
          at junit.extensions.TestSetup.run(TestSetup.java:23)

          FAILURES!!!

          I don't see BootLockMinion.log though. I will try to take a look later, but if you want to check it in with it disabled that would be fine.

          I also noticed when I tried to add the file I got:
          svn: Error resolving case of 'java\testing\org\apache\derbyTesting\functionTest
          'tests\store\BootLockTest.java

          That may be an environment issue on my part as I just started using cygwin instead of mks. I haven't looked at either issue closely.

          Show
          Kathey Marsden added a comment - I am guessing the old runwithj9=false because the spawned process stuff did not work with the old harness and j9, but I just tried the new test with weme6.1 and it failed with: 1) testBootLock(org.apache.derbyTesting.functionTests.tests.store.BootLockTest)j unit.framework.AssertionFailedError: Dual boot not detected: check BootLockMinio n.log at org.apache.derbyTesting.functionTests.tests.store.BootLockTest.testBo otLock(BootLockTest.java:92) at java.lang.reflect.AccessibleObject.invokeV(AccessibleObject.java:205) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java: 109) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57 ) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:22) at junit.extensions.TestSetup$1.protect(TestSetup.java:19) at junit.extensions.TestSetup.run(TestSetup.java:23) FAILURES!!! I don't see BootLockMinion.log though. I will try to take a look later, but if you want to check it in with it disabled that would be fine. I also noticed when I tried to add the file I got: svn: Error resolving case of 'java\testing\org\apache\derbyTesting\functionTest 'tests\store\BootLockTest.java That may be an environment issue on my part as I just started using cygwin instead of mks. I haven't looked at either issue closely.
          Hide
          Dag H. Wanvik added a comment -

          Thanks for testing this, Kathey! If there is no BootLockMinion.log, I guess the spawned program never got as far as booting Derby for some reason. You should be able to test the minion program standalone to see what happens, cf class Javadoc in BootLockMinion.
          That said, I have yet to test the new patch on Windows/cygwin, I'll get to that today.

          Show
          Dag H. Wanvik added a comment - Thanks for testing this, Kathey! If there is no BootLockMinion.log, I guess the spawned program never got as far as booting Derby for some reason. You should be able to test the minion program standalone to see what happens, cf class Javadoc in BootLockMinion. That said, I have yet to test the new patch on Windows/cygwin, I'll get to that today.
          Hide
          Dag H. Wanvik added a comment -

          Regressions passed on Solaris/Sun JDK 1.6, but I'll do one on Windows as well.

          Show
          Dag H. Wanvik added a comment - Regressions passed on Solaris/Sun JDK 1.6, but I'll do one on Windows as well.
          Hide
          Kathey Marsden added a comment -

          I think maybe I have the wrong patch. I see one was removed and then added with the same name after I downloaded it. I will get that one instead. What is the difference?

          Show
          Kathey Marsden added a comment - I think maybe I have the wrong patch. I see one was removed and then added with the same name after I downloaded it. I will get that one instead. What is the difference?
          Hide
          Kathey Marsden added a comment -

          Here is the error running BootLockMinion. Do we need to use EmbeddedSimpleDataSource instead?

          org.aache.derbyTesting.functionTests.tests.store.BootLockMinion wombat
          Exception in thread "main" java.lang.NoClassDefFoundError: javax.naming.Referenc
          eable
          at java.lang.ClassLoader.defineClassImpl(Native Method)
          at java.lang.ClassLoader.defineClass(ClassLoader.java:239)
          at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:10
          9)
          at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1073)
          at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1078)
          at java.net.URLClassLoader$LoadContext.run(URLClassLoader.java:570)
          at java.security.AccessController.doPrivileged(AccessController.java:227
          )
          at java.net.URLClassLoader.findClass(URLClassLoader.java:586)
          at com.ibm.oti.vm.URLSystemClassLoader.findClass(URLSystemClassLoader.ja
          va:26)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:641)
          at com.ibm.oti.vm.URLAppClassLoader.loadClass(URLAppClassLoader.java:146
          )
          at java.lang.ClassLoader.loadClass(ClassLoader.java:607)
          at java.lang.ClassLoader.defineClassImpl(Native Method)
          at java.lang.ClassLoader.defineClass(ClassLoader.java:239)
          at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:10
          9)
          at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1073)
          at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1078)
          at java.net.URLClassLoader$LoadContext.run(URLClassLoader.java:570)
          at java.security.AccessController.doPrivileged(AccessController.java:227
          )
          at java.net.URLClassLoader.findClass(URLClassLoader.java:586)
          at com.ibm.oti.vm.URLSystemClassLoader.findClass(URLSystemClassLoader.ja
          va:26)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:641)
          at com.ibm.oti.vm.URLAppClassLoader.loadClass(URLAppClassLoader.java:146
          )
          at java.lang.ClassLoader.loadClass(ClassLoader.java:607)
          at org.apache.derbyTesting.functionTests.tests.store.BootLockMinion.main
          (BootLockMinion.java:50)

          Show
          Kathey Marsden added a comment - Here is the error running BootLockMinion. Do we need to use EmbeddedSimpleDataSource instead? org.aache.derbyTesting.functionTests.tests.store.BootLockMinion wombat Exception in thread "main" java.lang.NoClassDefFoundError: javax.naming.Referenc eable at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:239) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:10 9) at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1073) at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1078) at java.net.URLClassLoader$LoadContext.run(URLClassLoader.java:570) at java.security.AccessController.doPrivileged(AccessController.java:227 ) at java.net.URLClassLoader.findClass(URLClassLoader.java:586) at com.ibm.oti.vm.URLSystemClassLoader.findClass(URLSystemClassLoader.ja va:26) at java.lang.ClassLoader.loadClass(ClassLoader.java:641) at com.ibm.oti.vm.URLAppClassLoader.loadClass(URLAppClassLoader.java:146 ) at java.lang.ClassLoader.loadClass(ClassLoader.java:607) at java.lang.ClassLoader.defineClassImpl(Native Method) at java.lang.ClassLoader.defineClass(ClassLoader.java:239) at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:10 9) at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1073) at java.net.URLClassLoader.findClassImpl(URLClassLoader.java:1078) at java.net.URLClassLoader$LoadContext.run(URLClassLoader.java:570) at java.security.AccessController.doPrivileged(AccessController.java:227 ) at java.net.URLClassLoader.findClass(URLClassLoader.java:586) at com.ibm.oti.vm.URLSystemClassLoader.findClass(URLSystemClassLoader.ja va:26) at java.lang.ClassLoader.loadClass(ClassLoader.java:641) at com.ibm.oti.vm.URLAppClassLoader.loadClass(URLAppClassLoader.java:146 ) at java.lang.ClassLoader.loadClass(ClassLoader.java:607) at org.apache.derbyTesting.functionTests.tests.store.BootLockMinion.main (BootLockMinion.java:50)
          Hide
          Dag H. Wanvik added a comment -

          Kathey Marsden (JIRA) writes:
          > [ https://issues.apache.org/jira/browse/DERBY-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864364#action_12864364 ]
          >
          > Kathey Marsden commented on DERBY-4179:
          > ---------------------------------------
          >

          > I think maybe I have the wrong patch. I see one was removed and
          > then added with the same name after I downloaded it. I will get
          > that one instead. What is the difference?

          I cleaned up some blankspace/tab stuff I had forgotten. Originally I
          made the patch via "svn rename" instead of "remove/add" of the minion
          program. But I think the second version of the patch has no sign of
          the svn rename op. I am not really sure of that could have tripped
          things up when yo you applied the patch.. I'll try it myself on cygwin
          now.

          Sorry for the inconvenience!

          Thanks,
          Dag

          Show
          Dag H. Wanvik added a comment - Kathey Marsden (JIRA) writes: > [ https://issues.apache.org/jira/browse/DERBY-4179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12864364#action_12864364 ] > > Kathey Marsden commented on DERBY-4179 : > --------------------------------------- > > I think maybe I have the wrong patch. I see one was removed and > then added with the same name after I downloaded it. I will get > that one instead. What is the difference? I cleaned up some blankspace/tab stuff I had forgotten. Originally I made the patch via "svn rename" instead of "remove/add" of the minion program. But I think the second version of the patch has no sign of the svn rename op. I am not really sure of that could have tripped things up when yo you applied the patch.. I'll try it myself on cygwin now. Sorry for the inconvenience! Thanks, Dag
          Hide
          Dag H. Wanvik added a comment -

          Thanks, Kathey! I guess this is a small device limitation. I'll try your suggestion!

          Show
          Dag H. Wanvik added a comment - Thanks, Kathey! I guess this is a small device limitation. I'll try your suggestion!
          Hide
          Dag H. Wanvik added a comment -

          Uploading derby-4179-junit-2, replacing earlier patches. This version uses EmbeddedSimpleDataSource in the minion program, which hopefully makes it run on a small device platform. I had no issues running the new test on Vista/cygwin.

          Note: I cut down the waiting time for the child to start and boot Derby from 30s to 15s in the interest of speed, our regressions are taking longer and longer. Is that too tight?

          Show
          Dag H. Wanvik added a comment - Uploading derby-4179-junit-2, replacing earlier patches. This version uses EmbeddedSimpleDataSource in the minion program, which hopefully makes it run on a small device platform. I had no issues running the new test on Vista/cygwin. Note: I cut down the waiting time for the child to start and boot Derby from 30s to 15s in the interest of speed, our regressions are taking longer and longer. Is that too tight?
          Hide
          Kathey Marsden added a comment -

          Thanks Dag. I think Knut's suggestion to check that it is not the process itself that failed is a good one. I am not exactly sure how you do that though and what exitValue() returns when something is still running.

          Show
          Kathey Marsden added a comment - Thanks Dag. I think Knut's suggestion to check that it is not the process itself that failed is a good one. I am not exactly sure how you do that though and what exitValue() returns when something is still running.
          Hide
          Kathey Marsden added a comment -

          Maybe instead it could just loop with 1 second waits for a period of time looking for the database lock file. That way it would wait just as long as needed. It might also help solve the problem of determining if BootLockMinion just didn't do its thing. Sometimes with JVM testing I know the tests are run with various options like turning JIT off which make things run very slowly and so it would be better to have the progress as soon as it is ready but wait 60 seconds total.

          Show
          Kathey Marsden added a comment - Maybe instead it could just loop with 1 second waits for a period of time looking for the database lock file. That way it would wait just as long as needed. It might also help solve the problem of determining if BootLockMinion just didn't do its thing. Sometimes with JVM testing I know the tests are run with various options like turning JIT off which make things run very slowly and so it would be better to have the progress as soon as it is ready but wait 60 seconds total.
          Hide
          Dag H. Wanvik added a comment -

          Thanks for that idea, Kathey! Meanwhile I went ahead and coded the parent to listen on a socket for the minion to be done, Since this test only runs embedded I can reuse the current configuration's server port, I think. So now the test runs in 2.5 seconds, maximally waiting for a minute. I am not sure, but I think CDC supports only datagram sockets, but I never open a stream, just listen for a connect, so hopefully it will work there as well. What do you think?

          Attaching derby-4179-junit-3 for this.

          Show
          Dag H. Wanvik added a comment - Thanks for that idea, Kathey! Meanwhile I went ahead and coded the parent to listen on a socket for the minion to be done, Since this test only runs embedded I can reuse the current configuration's server port, I think. So now the test runs in 2.5 seconds, maximally waiting for a minute. I am not sure, but I think CDC supports only datagram sockets, but I never open a stream, just listen for a connect, so hopefully it will work there as well. What do you think? Attaching derby-4179-junit-3 for this.
          Hide
          Dag H. Wanvik added a comment -

          As far as the exit value, I din't trust using that; after detroying the minion I saw an exit code of 143 on the Sun VM, but I don't know if that's standard or to be counted upon. Before the minion is killed I think exitValue will throw. In the new solution, the test fails if it doesn't hear from the minion program in 60 secs, so I think that should be sufficient hint.

          Show
          Dag H. Wanvik added a comment - As far as the exit value, I din't trust using that; after detroying the minion I saw an exit code of 143 on the Sun VM, but I don't know if that's standard or to be counted upon. Before the minion is killed I think exitValue will throw. In the new solution, the test fails if it doesn't hear from the minion program in 60 secs, so I think that should be sufficient hint.
          Hide
          Kathey Marsden added a comment -

          Still failing with weme for some reason with the Minion did not start message. The trick of running BootLockMinion doesn't work though because we don't have a server socket. I will debug a bit and try to dump the process output.

          Show
          Kathey Marsden added a comment - Still failing with weme for some reason with the Minion did not start message. The trick of running BootLockMinion doesn't work though because we don't have a server socket. I will debug a bit and try to dump the process output.
          Hide
          Kathey Marsden added a comment -

          Well I guess we aren't as set up to spawn a j9 process as I thought.

          The error is actually
          .JVMJ9VM011W Unable to load jclfoun10_24: The specified module could not be foun
          d.
          JVMEXEX013E Internal VM error: Failed to create Java VM
          JVMEXEX014I Run C:\cygwin\ibmsvn\ntsoftware\weme6.2\bin\j9.exe -help for usage

          execJavaProcess does pick up the j9 executable but does not pass on the other settings.

          This is how my script invokes the test with j9. It probably has a lot of legacy system properties not needed, but I suppose execJavaCmd should just pass along all system properties, but I don't know how it would get the bootclasspath. Perhaps -Dbootcp was a way to pass it on in the old harness.

          c:/cygwin/ibmsvn/ntsoftware/weme6.2/bin/j9 -jcl:foun11 -DderbyTesting.serverho
          st=localhost -DderbyTesting.clienthost=localhost -Demma.active= -Xbootclasspath/
          a:c:/cygwin/ibmsvn/ntsoftware/weme6.2/lib/jdbc.jar -Dbootcp=c:/cygwin/ibmsvn/nts
          oftware/weme6.2/lib/jdbc.jar junit.textui.TestRunner org.apache.derbyTesting.fun
          ctionTests.tests.store.BootLockTest

          Anyway I best get back to my 10.6 testing. Disabling for J2ME for now and and adding an issue to enable it later would be fine by me. It might be convenient to have the code below to print the error output on error.

          ....
          catch (SocketTimeoutException e) {
          p.destroy();
          p.waitFor();
          // Print error output
          BufferedReader er = new BufferedReader(new InputStreamReader(p.getErrorStream()));
          String failmsg = "Minion did not start or boot db in 60 seconds\n";
          String erout= null ;
          do

          { erout = er.readLine(); if (erout != null) failmsg += erout; }

          while (erout != null);
          fail(failmsg);
          }

          Show
          Kathey Marsden added a comment - Well I guess we aren't as set up to spawn a j9 process as I thought. The error is actually .JVMJ9VM011W Unable to load jclfoun10_24: The specified module could not be foun d. JVMEXEX013E Internal VM error: Failed to create Java VM JVMEXEX014I Run C:\cygwin\ibmsvn\ntsoftware\weme6.2\bin\j9.exe -help for usage execJavaProcess does pick up the j9 executable but does not pass on the other settings. This is how my script invokes the test with j9. It probably has a lot of legacy system properties not needed, but I suppose execJavaCmd should just pass along all system properties, but I don't know how it would get the bootclasspath. Perhaps -Dbootcp was a way to pass it on in the old harness. c:/cygwin/ibmsvn/ntsoftware/weme6.2/bin/j9 -jcl:foun11 -DderbyTesting.serverho st=localhost -DderbyTesting.clienthost=localhost -Demma.active= -Xbootclasspath/ a:c:/cygwin/ibmsvn/ntsoftware/weme6.2/lib/jdbc.jar -Dbootcp=c:/cygwin/ibmsvn/nts oftware/weme6.2/lib/jdbc.jar junit.textui.TestRunner org.apache.derbyTesting.fun ctionTests.tests.store.BootLockTest Anyway I best get back to my 10.6 testing. Disabling for J2ME for now and and adding an issue to enable it later would be fine by me. It might be convenient to have the code below to print the error output on error. .... catch (SocketTimeoutException e) { p.destroy(); p.waitFor(); // Print error output BufferedReader er = new BufferedReader(new InputStreamReader(p.getErrorStream())); String failmsg = "Minion did not start or boot db in 60 seconds\n"; String erout= null ; do { erout = er.readLine(); if (erout != null) failmsg += erout; } while (erout != null); fail(failmsg); }
          Hide
          Dag H. Wanvik added a comment -

          Thanks again, Kathey! I tried to run the test on the Sun phoneME, it forked the minion all right, but the
          parent didn't experience an error connecting. I just saw the warning XSDB7 on derby.log instead. apparently because of
          StorageFile.NO_FILE_LOCK_SUPPORT.

          So, the test would have to be changed for phoneMe in any case, maybe by setting derby.database.forceDatabaseLock.

          I'll just disable for now for J2ME as you suggest.

          Show
          Dag H. Wanvik added a comment - Thanks again, Kathey! I tried to run the test on the Sun phoneME, it forked the minion all right, but the parent didn't experience an error connecting. I just saw the warning XSDB7 on derby.log instead. apparently because of StorageFile.NO_FILE_LOCK_SUPPORT. So, the test would have to be changed for phoneMe in any case, maybe by setting derby.database.forceDatabaseLock. I'll just disable for now for J2ME as you suggest.
          Hide
          Kathey Marsden added a comment -

          Are you saying that with phoneME you can actually dual boot? I have always only used weme on Windows where it is ok, but should probably try a linux version if I can find one if this is the case.

          Show
          Kathey Marsden added a comment - Are you saying that with phoneME you can actually dual boot? I have always only used weme on Windows where it is ok, but should probably try a linux version if I can find one if this is the case.
          Hide
          Dag H. Wanvik added a comment -

          Yes, I am. For phoneME, we don't use DirStorageFactory4, which again uses DirFile4, which contains the working code for file locking. Cf. the test ca line 1693 in BaseMonitor.java.

          DirFile#getExclusiveFileLock which is used instead of DirFile4, always returns NO_FILE_LOCK_SUPPORT.

          I tried letting phoneME run with the *4 classes, but Foundation 1.1 [*] seems to lack the NIO classes necessary, notably the crucial FileChannel.lock method used by DirFile4#getExclusiveFileLock.

          On the bright side, I guess dual booting on constrained device is a less likely scenario?

          [*] http://java.sun.com/javame/reference/apis/jsr219/

          Show
          Dag H. Wanvik added a comment - Yes, I am. For phoneME, we don't use DirStorageFactory4, which again uses DirFile4, which contains the working code for file locking. Cf. the test ca line 1693 in BaseMonitor.java. DirFile#getExclusiveFileLock which is used instead of DirFile4, always returns NO_FILE_LOCK_SUPPORT. I tried letting phoneME run with the *4 classes, but Foundation 1.1 [*] seems to lack the NIO classes necessary, notably the crucial FileChannel.lock method used by DirFile4#getExclusiveFileLock. On the bright side, I guess dual booting on constrained device is a less likely scenario? [*] http://java.sun.com/javame/reference/apis/jsr219/
          Hide
          Dag H. Wanvik added a comment -

          Filed DERBY-4646 as an improvement issue for the lacking dual boot protection on ME. I guess it could qualify as a bug as well..

          Show
          Dag H. Wanvik added a comment - Filed DERBY-4646 as an improvement issue for the lacking dual boot protection on ME. I guess it could qualify as a bug as well..
          Hide
          Dag H. Wanvik added a comment - - edited

          Uploading version 4 of this test, please review.
          Passes for me on Windows, Sun phoneME and Solaris/Linux.
          Rerunning regressions now.

          Details:

          • Rewrite of boot lock test to JUnit, this also solves the problem
            with running with jars in relative classpaths.
          • Added test of effectiveness of derby.database.forceDatabaseLock for
            phoneME platforms, until we implement DERBY-4646. (I could not find
            any other such test).
          • Fixed JUnit harness to correctly fork a Sun phoneME vm (image name
            is cvm, not java),
          • Added logic to capture what happens to minion (if it fails) by
            capturing its stderr (if it ever gets started, that is), cf. example enclosed.
          • Added socket logic to communicate to parent when minion has booted
            the "dual boot" candidate to avoid having to wait for 30 seconds or
            more to be sure it has done so. Test now runs in a few seconds.
          • Skips test for j9 for now, since this platform fails on the fork
            operation according to Kathey. I suggest we open a new JIRA to fix
            this if desired.

          Example of the minion failing:
          -----------------------------
          There was 1 failure:
          1) testBootLock(org.apache.derbyTesting.functionTests.tests.store.BootLockTest)junit.framework.AssertionFailedError: Minion did not start or boot db in 60 seconds.
          ----Minion's stderr:
          java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Thread.java:1206) at org.apache.derbyTesting.functionTests.tests.store.BootLockMinion.main(BootLockMinion.java:48)
          ----Minion's stderr ended
          at org.apache.derbyTesting.functionTests.tests.store.BootLockTest.testBootLock(BootLockTest.java:201)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
          at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)
          at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24)
          at junit.extensions.TestSetup$1.protect(TestSetup.java:21)
          at junit.extensions.TestSetup.run(TestSetup.java:25)

          FAILURES!!!
          Tests run: 1, Failures: 1, Errors: 0

          Show
          Dag H. Wanvik added a comment - - edited Uploading version 4 of this test, please review. Passes for me on Windows, Sun phoneME and Solaris/Linux. Rerunning regressions now. Details: Rewrite of boot lock test to JUnit, this also solves the problem with running with jars in relative classpaths. Added test of effectiveness of derby.database.forceDatabaseLock for phoneME platforms, until we implement DERBY-4646 . (I could not find any other such test). Fixed JUnit harness to correctly fork a Sun phoneME vm (image name is cvm, not java), Added logic to capture what happens to minion (if it fails) by capturing its stderr (if it ever gets started, that is), cf. example enclosed. Added socket logic to communicate to parent when minion has booted the "dual boot" candidate to avoid having to wait for 30 seconds or more to be sure it has done so. Test now runs in a few seconds. Skips test for j9 for now, since this platform fails on the fork operation according to Kathey. I suggest we open a new JIRA to fix this if desired. Example of the minion failing: ----------------------------- There was 1 failure: 1) testBootLock(org.apache.derbyTesting.functionTests.tests.store.BootLockTest)junit.framework.AssertionFailedError: Minion did not start or boot db in 60 seconds. ----Minion's stderr: java.lang.Exception: Stack trace at java.lang.Thread.dumpStack(Thread.java:1206) at org.apache.derbyTesting.functionTests.tests.store.BootLockMinion.main(BootLockMinion.java:48) ----Minion's stderr ended at org.apache.derbyTesting.functionTests.tests.store.BootLockTest.testBootLock(BootLockTest.java:201) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:109) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at org.apache.derbyTesting.junit.BaseTestSetup.run(BaseTestSetup.java:57) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) at junit.extensions.TestDecorator.basicRun(TestDecorator.java:24) at junit.extensions.TestSetup$1.protect(TestSetup.java:21) at junit.extensions.TestSetup.run(TestSetup.java:25) FAILURES!!! Tests run: 1, Failures: 1, Errors: 0
          Hide
          Kathey Marsden added a comment -

          I probably won't have time to look at the final patch, but it all sounds great based on previous looks and the changes you describe. I'd say go ahead and commit. I will file an issue regarding spawning the process with weme

          Show
          Kathey Marsden added a comment - I probably won't have time to look at the final patch, but it all sounds great based on previous looks and the changes you describe. I'd say go ahead and commit. I will file an issue regarding spawning the process with weme
          Hide
          Dag H. Wanvik added a comment -

          Thanks for your help with this one, Kathey!

          Show
          Dag H. Wanvik added a comment - Thanks for your help with this one, Kathey!
          Hide
          Dag H. Wanvik added a comment -

          Uploaded version #5, which contains some small improvements relative to version 4.

          Show
          Dag H. Wanvik added a comment - Uploaded version #5, which contains some small improvements relative to version 4.
          Hide
          Dag H. Wanvik added a comment - - edited

          Committed derby-4179-junit-5 as svn 942286 and some doc/comment fixes in svn 942476 and 942480.

          Show
          Dag H. Wanvik added a comment - - edited Committed derby-4179-junit-5 as svn 942286 and some doc/comment fixes in svn 942476 and 942480.
          Hide
          Dag H. Wanvik added a comment -

          Resolving, since the original failing test is gone, even though the root cause may have been different.

          Show
          Dag H. Wanvik added a comment - Resolving, since the original failing test is gone, even though the root cause may have been different.
          Hide
          Knut Anders Hatlen added a comment -

          + // Can't use the DatabasePropertyTestSetup.singleProperty, since
          + // that method sets a database property (not a system property),
          + // and the minion is the one creating the database here. An
          + // alternative would be to let minion set it.

          Would SystemPropertyTestSetup be an alternative?

          Show
          Knut Anders Hatlen added a comment - + // Can't use the DatabasePropertyTestSetup.singleProperty, since + // that method sets a database property (not a system property), + // and the minion is the one creating the database here. An + // alternative would be to let minion set it. Would SystemPropertyTestSetup be an alternative?
          Hide
          Dag H. Wanvik added a comment -

          Ah, thanks, missed that one I was considering writing one.. this saves me that effort!

          Show
          Dag H. Wanvik added a comment - Ah, thanks, missed that one I was considering writing one.. this saves me that effort!
          Hide
          Dag H. Wanvik added a comment -

          Committed patch svn 942587 which replaces the home-grown system property setup/teardown code with SystemPropertyTestSetup.

          Show
          Dag H. Wanvik added a comment - Committed patch svn 942587 which replaces the home-grown system property setup/teardown code with SystemPropertyTestSetup.
          Hide
          Kathey Marsden added a comment -

          Backported to 10.6 and 10.5

          Show
          Kathey Marsden added a comment - Backported to 10.6 and 10.5

            People

            • Assignee:
              Dag H. Wanvik
              Reporter:
              Kathey Marsden
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development