Derby
  1. Derby
  2. DERBY-5100

GetCurrentPropertiesTest depends on implicit ordering of test cases

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 10.8.1.2
    • Fix Version/s: 10.8.1.2
    • Component/s: Test
    • Labels:
      None
    • Bug behavior facts:
      Regression Test Failure

      Description

      The test cases in GetCurrentPropertiesTest must run in the order in which they are defined in the source file. If for example the test case testPropertiesTraceOn runs first, the test will fail with the following stack trace:

      1) testPropertiesTraceOn(org.apache.derbyTesting.functionTests.tests.derbynet.GetCurrentPropertiesTest)junit.framework.ComparisonFailure: expected:<[tru]e> but was:<[fals]e>
      at org.apache.derbyTesting.functionTests.tests.derbynet.GetCurrentPropertiesTest.testPropertiesTraceOn(GetCurrentPropertiesTest.java:206)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at org.apache.derbyTesting.junit.BaseTestCase.runBare(BaseTestCase.java:112)
      (...)

      Seen in the nightly tests, for example here: http://dbtg.foundry.sun.com/derby/test/Daily/jvm1.7/testing/testlog/vista-64/1076682-suitesAll_diff.txt

      1. derby-5100-1a.diff
        5 kB
        Knut Anders Hatlen

        Issue Links

          Activity

          Hide
          Knut Anders Hatlen added a comment -

          Committed revision 1082226.

          Show
          Knut Anders Hatlen added a comment - Committed revision 1082226.
          Hide
          Knut Anders Hatlen added a comment -

          The attached patch adds a utility method called orderedSuite() to TestConfiguration. The method takes a class argument and returns a test suite that runs all the test methods found in the class in lexicographic order.

          The patch also makes GetCurrentPropertiesTest use the method, and renames the test methods in the class so that their lexicographic order is the same as the order in which they appear in the source file.

          This change made the test run reliably on Java 7 for me.

          Show
          Knut Anders Hatlen added a comment - The attached patch adds a utility method called orderedSuite() to TestConfiguration. The method takes a class argument and returns a test suite that runs all the test methods found in the class in lexicographic order. The patch also makes GetCurrentPropertiesTest use the method, and renames the test methods in the class so that their lexicographic order is the same as the order in which they appear in the source file. This change made the test run reliably on Java 7 for me.
          Hide
          Knut Anders Hatlen added a comment -

          Some of the test methods in this test (and also in NetworkServerControlApiTest - see DERBY-5115) depend on the connections having a certain connection number. This is only possible to guarantee if the test methods are invoked in a particular order, or if we restart the network server between each test method. I suggest that we explicitly order the methods in this test and add a comment saying why the order is required. I'll add a utility method that sorts the tests alphabetically, and then we can add a serial number to the test names to define the order (test_01_blahblah, test_02_blahblah, ...).

          Show
          Knut Anders Hatlen added a comment - Some of the test methods in this test (and also in NetworkServerControlApiTest - see DERBY-5115 ) depend on the connections having a certain connection number. This is only possible to guarantee if the test methods are invoked in a particular order, or if we restart the network server between each test method. I suggest that we explicitly order the methods in this test and add a comment saying why the order is required. I'll add a utility method that sorts the tests alphabetically, and then we can add a serial number to the test names to define the order (test_01_blahblah, test_02_blahblah, ...).

            People

            • Assignee:
              Knut Anders Hatlen
              Reporter:
              Knut Anders Hatlen
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development