Derby
  1. Derby
  2. DERBY-4638

Expose names of top-level test suites for easier scripting

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 10.6.2.1, 10.7.1.1
    • Fix Version/s: 10.6.2.1, 10.7.1.1
    • Component/s: Test
    • Labels:
      None

      Description

      I sometimes run each of the top-level JUnit test suites individually so that they can be run in parallel and complete faster. It would be useful if AllPackages could expose the list of suites to allow scripting of such tasks without needing to change the script when a new suite is added.

      1. DerbyTestRunner.tar.gz
        13 kB
        Knut Anders Hatlen
      2. list-tests.diff
        13 kB
        Knut Anders Hatlen

        Issue Links

          Activity

          Hide
          Knut Anders Hatlen added a comment -

          Thanks for checking out the possibility for a backport, Kristian. Since there's not more to do on this issue, I'm closing it.

          Show
          Knut Anders Hatlen added a comment - Thanks for checking out the possibility for a backport, Kristian. Since there's not more to do on this issue, I'm closing it.
          Hide
          Kristian Waagan added a comment -

          I tried back-porting this to 10.5, but there are some issues with that, besides the merges not being clean;
          1) Configurable port functionality missing (fixed by back-porting DERBY-4217)
          2) SysinfoTest fails (fixed by DERBY-858?)

          It may be possible to do this, but I guess the safest approach is to have more patience when testing changes on the 10.5 branch

          In any case, thanks a lot to Tiago and Knut Anders for making this improvement happen!

          Show
          Kristian Waagan added a comment - I tried back-porting this to 10.5, but there are some issues with that, besides the merges not being clean; 1) Configurable port functionality missing (fixed by back-porting DERBY-4217 ) 2) SysinfoTest fails (fixed by DERBY-858 ?) It may be possible to do this, but I guess the safest approach is to have more patience when testing changes on the 10.5 branch In any case, thanks a lot to Tiago and Knut Anders for making this improvement happen!
          Hide
          Lily Wei added a comment -

          Thank you for this wonderful addition + for testing faster and throughly. It is this kind of long term thinking that makes Derby better and better everyday. Really enjoy it.

          Show
          Lily Wei added a comment - Thank you for this wonderful addition + for testing faster and throughly. It is this kind of long term thinking that makes Derby better and better everyday. Really enjoy it.
          Hide
          Dag H. Wanvik added a comment -

          Thanks for this useful addition + the parallel tester bundle, having fun trying to make my regressions run fast, here
          Nice to get some benefit from the work that's been done to make the ports configurable!

          Show
          Dag H. Wanvik added a comment - Thanks for this useful addition + the parallel tester bundle, having fun trying to make my regressions run fast, here Nice to get some benefit from the work that's been done to make the ports configurable!
          Hide
          Knut Anders Hatlen added a comment -

          Merged to the 10.6 branch and committed revision 942049.

          Show
          Knut Anders Hatlen added a comment - Merged to the 10.6 branch and committed revision 942049.
          Hide
          Knut Anders Hatlen added a comment -

          Hi Myrna,
          It should be safe to backport it to 10.6. I'll look into doing that.

          Show
          Knut Anders Hatlen added a comment - Hi Myrna, It should be safe to backport it to 10.6. I'll look into doing that.
          Hide
          Myrna van Lunteren added a comment -

          Can this be backported to 10.6?

          Show
          Myrna van Lunteren added a comment - Can this be backported to 10.6?
          Hide
          Knut Anders Hatlen added a comment -

          Hi Bryan,

          That sounds like a good idea. I haven't used Ant a lot for running JUnit tests, but I don't see any reason why it shouldn't be possible to do something similar in Ant.

          Show
          Knut Anders Hatlen added a comment - Hi Bryan, That sounds like a good idea. I haven't used Ant a lot for running JUnit tests, but I don't see any reason why it shouldn't be possible to do something similar in Ant.
          Hide
          Bryan Pendleton added a comment -

          Can we make an Ant script which does this, by using the parallel task support in Ant?

          Show
          Bryan Pendleton added a comment - Can we make an Ant script which does this, by using the parallel task support in Ant?
          Hide
          Knut Anders Hatlen added a comment -

          I threw together an application that extracts the list of suites from the new public method in AllPackages and runs them in parallel. On my machine, suites.All takes about 38 minutes when I run the suites in four sub-processes this way. Attaching it here in case someone is interested in using it as a starting point. There's a README.txt file in the tarball explaining how to use it.

          Show
          Knut Anders Hatlen added a comment - I threw together an application that extracts the list of suites from the new public method in AllPackages and runs them in parallel. On my machine, suites.All takes about 38 minutes when I run the suites in four sub-processes this way. Attaching it here in case someone is interested in using it as a starting point. There's a README.txt file in the tarball explaining how to use it.
          Hide
          Knut Anders Hatlen added a comment -

          Committed revision 941627.

          Show
          Knut Anders Hatlen added a comment - Committed revision 941627.
          Hide
          Knut Anders Hatlen added a comment -

          I have run suites.All on Java 6 and on Java 1.4.2 and verified that all the tests still run, and that the suites that need to be added using reflection on 1.4.2 (because they are compiled with target level 1.5 or higher) don't cause any problems with these changes.

          I've also run the replication test on phoneME to verify that moving the check of the JDBC level didn't cause problems. Before the change, the suite() method failed with a NoClassDefFoundError. After the change, suite() returned successfully (but didn't add any tests, as expected).

          Show
          Knut Anders Hatlen added a comment - I have run suites.All on Java 6 and on Java 1.4.2 and verified that all the tests still run, and that the suites that need to be added using reflection on 1.4.2 (because they are compiled with target level 1.5 or higher) don't cause any problems with these changes. I've also run the replication test on phoneME to verify that moving the check of the JDBC level didn't cause problems. Before the change, the suite() method failed with a NoClassDefFoundError. After the change, suite() returned successfully (but didn't add any tests, as expected).
          Hide
          Knut Anders Hatlen added a comment -

          The attached patch makes it possible to retrieve the list of test suites either by calling the method AllPackages.getTopLevelSuiteNames(), which returns an array of strings with class names, or by running AllPackages from the command line:

          $ java -cp classes:tools/java/junit.jar org.apache.derbyTesting.functionTests.suites.AllPackages
          org.apache.derbyTesting.functionTests.tests.derbynet._Suite
          org.apache.derbyTesting.functionTests.tests.lang._Suite
          org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite
          org.apache.derbyTesting.functionTests.tests.store._Suite
          org.apache.derbyTesting.functionTests.tests.tools._Suite
          org.apache.derbyTesting.functionTests.tests.engine._Suite
          org.apache.derbyTesting.functionTests.tests.demo._Suite
          org.apache.derbyTesting.functionTests.tests.memory._Suite
          org.apache.derbyTesting.functionTests.tests.memorydb._Suite
          org.apache.derbyTesting.functionTests.tests.i18n._Suite
          org.apache.derbyTesting.functionTests.tests.multi.StressMultiTest
          org.apache.derbyTesting.functionTests.tests.jdbc4._Suite
          org.apache.derbyTesting.functionTests.tests.management._Suite
          org.apache.derbyTesting.unitTests.junit._Suite
          org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite
          org.apache.derbyTesting.functionTests.suites.EncryptionSuite
          org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite

          The patch makes the following changes:

          suites/AllPackages.java:

          • Added public methods getTopLevelSuiteNames() and main() to extract the list of test suites

          suites/All.java:

          • Moved adding of EncryptionSuite and ReplicationSuite to AllPackages so that they also are included in the list of test suites

          tests/replicationTests/ReplicationSuite.java:

          • Made suite() check vmSupportsJDBC3() before adding test cases. Before, this was done in suites.All.suite(), but when the adding of ReplicationSuite was moved from All to AllPackages, it was also changed to be consistent with the other test suites, which do these checks locally in their suite() methods.
          Show
          Knut Anders Hatlen added a comment - The attached patch makes it possible to retrieve the list of test suites either by calling the method AllPackages.getTopLevelSuiteNames(), which returns an array of strings with class names, or by running AllPackages from the command line: $ java -cp classes:tools/java/junit.jar org.apache.derbyTesting.functionTests.suites.AllPackages org.apache.derbyTesting.functionTests.tests.derbynet._Suite org.apache.derbyTesting.functionTests.tests.lang._Suite org.apache.derbyTesting.functionTests.tests.jdbcapi._Suite org.apache.derbyTesting.functionTests.tests.store._Suite org.apache.derbyTesting.functionTests.tests.tools._Suite org.apache.derbyTesting.functionTests.tests.engine._Suite org.apache.derbyTesting.functionTests.tests.demo._Suite org.apache.derbyTesting.functionTests.tests.memory._Suite org.apache.derbyTesting.functionTests.tests.memorydb._Suite org.apache.derbyTesting.functionTests.tests.i18n._Suite org.apache.derbyTesting.functionTests.tests.multi.StressMultiTest org.apache.derbyTesting.functionTests.tests.jdbc4._Suite org.apache.derbyTesting.functionTests.tests.management._Suite org.apache.derbyTesting.unitTests.junit._Suite org.apache.derbyTesting.functionTests.tests.upgradeTests._Suite org.apache.derbyTesting.functionTests.suites.EncryptionSuite org.apache.derbyTesting.functionTests.tests.replicationTests.ReplicationSuite The patch makes the following changes: suites/AllPackages.java: Added public methods getTopLevelSuiteNames() and main() to extract the list of test suites suites/All.java: Moved adding of EncryptionSuite and ReplicationSuite to AllPackages so that they also are included in the list of test suites tests/replicationTests/ReplicationSuite.java: Made suite() check vmSupportsJDBC3() before adding test cases. Before, this was done in suites.All.suite(), but when the adding of ReplicationSuite was moved from All to AllPackages, it was also changed to be consistent with the other test suites, which do these checks locally in their suite() methods.

            People

            • Assignee:
              Knut Anders Hatlen
              Reporter:
              Knut Anders Hatlen
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development