Lucene - Core
  1. Lucene - Core
  2. LUCENE-3995

In LuceneTestCase.beforeClass, make a new random (also using the class hashcode) to vary defaults

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: 4.0-ALPHA
    • Component/s: general/test
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      In LuceneTestCase, we set many static defaults like:

      • default codec
      • default infostream impl
      • default locale
      • default timezone
      • default similarity

      Currently each test run gets a single seed for the run, which means for example across one test run
      every single test will have say, SimpleText + infostream=off + Locale=german + timezone=EDT + similarity=BM25

      Because of that, we lose lots of basic mixed coverage across tests, and it also means the unfortunate
      individual who gets SimpleText or other slow options gets a REALLY SLOW test run, rather than amortizing
      this across all test runs.

      We should at least make a new random (getRandom() ^ className.hashCode()) to fix this so it works like before,
      but unfortunately that only fixes it for LuceneTestCase.

      Won't any subclasses that make random decisions in @BeforeClass (and we have many) still have the same problem?
      Maybe RandomizedRunner can instead be improved here?

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          19h 35m 1 Dawid Weiss 18/Apr/12 17:04
          Resolved Resolved Closed Closed
          386d 18h 38m 1 Uwe Schindler 10/May/13 11:42
          Uwe Schindler made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Dawid Weiss made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 4.0 [ 12314025 ]
          Resolution Fixed [ 1 ]
          Hide
          Dawid Weiss added a comment -

          Fixed in trunk.

          Show
          Dawid Weiss added a comment - Fixed in trunk.
          Hide
          Robert Muir added a comment -

          yeah that looks right to me. I think currently randomVal will stay the same.

          Show
          Robert Muir added a comment - yeah that looks right to me. I think currently randomVal will stay the same.
          Hide
          Dawid Weiss added a comment -

          Robert, this would mean it works fine, right (note dumped randomVal for each suite)?

          Executing 296 suites with 4 JVMs.
          Suite: org.apache.lucene.util.TestCloseableThreadLocal
             > (@BeforeClass output)
            1> randomVal: 9
            1> 
          
          OK      0.05s J1 | TestCloseableThreadLocal.testDefaultValueWithoutSetting
          OK      0.01s J1 | TestCloseableThreadLocal.testInitValue
          OK      0.01s J1 | TestCloseableThreadLocal.testNullValue
          Completed on J1 in 0.27s, 3 tests
           
          Suite: org.apache.lucene.util.TestTwoPhaseCommitTool
             > (@BeforeClass output)
            1> randomVal: 6
            1> 
          
          OK      0.04s J2 | TestTwoPhaseCommitTool.testRollback
          OK      0.01s J2 | TestTwoPhaseCommitTool.testNullTPCs
          OK      0.01s J2 | TestTwoPhaseCommitTool.testWrapper
          OK      0.01s J2 | TestTwoPhaseCommitTool.testPrepareThenCommit
          Completed on J2 in 0.37s, 4 tests
           
          Suite: org.apache.lucene.util.TestNamedSPILoader
             > (@BeforeClass output)
            1> randomVal: 7
            1> 
          
          OK      0.04s J0 | TestNamedSPILoader.testAvailableServices
          OK      0.01s J0 | TestNamedSPILoader.testBogusLookup
          OK      0.01s J0 | TestNamedSPILoader.testLookup
          Completed on J0 in 0.34s, 3 tests
           
          Suite: org.apache.lucene.util.TestSmallFloat
             > (@BeforeClass output)
            1> randomVal: 2
            1> 
          
          OK      0.20s J3 | TestSmallFloat.testFloatToByte
          OK      0.01s J3 | TestSmallFloat.testByteToFloat
          Completed on J3 in 0.48s, 2 tests
           
          Suite: org.apache.lucene.index.TestTerm
             > (@BeforeClass output)
            1> randomVal: 0
            1>  
          
          Show
          Dawid Weiss added a comment - Robert, this would mean it works fine, right (note dumped randomVal for each suite)? Executing 296 suites with 4 JVMs. Suite: org.apache.lucene.util.TestCloseableThreadLocal > (@BeforeClass output) 1> randomVal: 9 1> OK 0.05s J1 | TestCloseableThreadLocal.testDefaultValueWithoutSetting OK 0.01s J1 | TestCloseableThreadLocal.testInitValue OK 0.01s J1 | TestCloseableThreadLocal.testNullValue Completed on J1 in 0.27s, 3 tests Suite: org.apache.lucene.util.TestTwoPhaseCommitTool > (@BeforeClass output) 1> randomVal: 6 1> OK 0.04s J2 | TestTwoPhaseCommitTool.testRollback OK 0.01s J2 | TestTwoPhaseCommitTool.testNullTPCs OK 0.01s J2 | TestTwoPhaseCommitTool.testWrapper OK 0.01s J2 | TestTwoPhaseCommitTool.testPrepareThenCommit Completed on J2 in 0.37s, 4 tests Suite: org.apache.lucene.util.TestNamedSPILoader > (@BeforeClass output) 1> randomVal: 7 1> OK 0.04s J0 | TestNamedSPILoader.testAvailableServices OK 0.01s J0 | TestNamedSPILoader.testBogusLookup OK 0.01s J0 | TestNamedSPILoader.testLookup Completed on J0 in 0.34s, 3 tests Suite: org.apache.lucene.util.TestSmallFloat > (@BeforeClass output) 1> randomVal: 2 1> OK 0.20s J3 | TestSmallFloat.testFloatToByte OK 0.01s J3 | TestSmallFloat.testByteToFloat Completed on J3 in 0.48s, 2 tests Suite: org.apache.lucene.index.TestTerm > (@BeforeClass output) 1> randomVal: 0 1>
          Hide
          Dawid Weiss added a comment -

          Note to myself - this also affectes test coverage because it reduces static context entropy (as pointed by Robert, Uwe).

          Show
          Dawid Weiss added a comment - Note to myself - this also affectes test coverage because it reduces static context entropy (as pointed by Robert, Uwe).
          Dawid Weiss made changes -
          Link This issue relates to LUCENE-3993 [ LUCENE-3993 ]
          Dawid Weiss made changes -
          Field Original Value New Value
          Assignee Dawid Weiss [ dweiss ]
          Robert Muir created issue -

            People

            • Assignee:
              Dawid Weiss
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development