Lucene - Core
  1. Lucene - Core
  2. LUCENE-3995

In LuceneTestCase.beforeClass, make a new random (also using the class hashcode) to vary defaults

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: 4.0-ALPHA
    • Component/s: general/test
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      In LuceneTestCase, we set many static defaults like:

      • default codec
      • default infostream impl
      • default locale
      • default timezone
      • default similarity

      Currently each test run gets a single seed for the run, which means for example across one test run
      every single test will have say, SimpleText + infostream=off + Locale=german + timezone=EDT + similarity=BM25

      Because of that, we lose lots of basic mixed coverage across tests, and it also means the unfortunate
      individual who gets SimpleText or other slow options gets a REALLY SLOW test run, rather than amortizing
      this across all test runs.

      We should at least make a new random (getRandom() ^ className.hashCode()) to fix this so it works like before,
      but unfortunately that only fixes it for LuceneTestCase.

      Won't any subclasses that make random decisions in @BeforeClass (and we have many) still have the same problem?
      Maybe RandomizedRunner can instead be improved here?

        Issue Links

          Activity

          Robert Muir created issue -
          Dawid Weiss made changes -
          Field Original Value New Value
          Assignee Dawid Weiss [ dweiss ]
          Dawid Weiss made changes -
          Link This issue relates to LUCENE-3993 [ LUCENE-3993 ]
          Hide
          Dawid Weiss added a comment -

          Note to myself - this also affectes test coverage because it reduces static context entropy (as pointed by Robert, Uwe).

          Show
          Dawid Weiss added a comment - Note to myself - this also affectes test coverage because it reduces static context entropy (as pointed by Robert, Uwe).
          Hide
          Dawid Weiss added a comment -

          Robert, this would mean it works fine, right (note dumped randomVal for each suite)?

          Executing 296 suites with 4 JVMs.
          Suite: org.apache.lucene.util.TestCloseableThreadLocal
             > (@BeforeClass output)
            1> randomVal: 9
            1> 
          
          OK      0.05s J1 | TestCloseableThreadLocal.testDefaultValueWithoutSetting
          OK      0.01s J1 | TestCloseableThreadLocal.testInitValue
          OK      0.01s J1 | TestCloseableThreadLocal.testNullValue
          Completed on J1 in 0.27s, 3 tests
           
          Suite: org.apache.lucene.util.TestTwoPhaseCommitTool
             > (@BeforeClass output)
            1> randomVal: 6
            1> 
          
          OK      0.04s J2 | TestTwoPhaseCommitTool.testRollback
          OK      0.01s J2 | TestTwoPhaseCommitTool.testNullTPCs
          OK      0.01s J2 | TestTwoPhaseCommitTool.testWrapper
          OK      0.01s J2 | TestTwoPhaseCommitTool.testPrepareThenCommit
          Completed on J2 in 0.37s, 4 tests
           
          Suite: org.apache.lucene.util.TestNamedSPILoader
             > (@BeforeClass output)
            1> randomVal: 7
            1> 
          
          OK      0.04s J0 | TestNamedSPILoader.testAvailableServices
          OK      0.01s J0 | TestNamedSPILoader.testBogusLookup
          OK      0.01s J0 | TestNamedSPILoader.testLookup
          Completed on J0 in 0.34s, 3 tests
           
          Suite: org.apache.lucene.util.TestSmallFloat
             > (@BeforeClass output)
            1> randomVal: 2
            1> 
          
          OK      0.20s J3 | TestSmallFloat.testFloatToByte
          OK      0.01s J3 | TestSmallFloat.testByteToFloat
          Completed on J3 in 0.48s, 2 tests
           
          Suite: org.apache.lucene.index.TestTerm
             > (@BeforeClass output)
            1> randomVal: 0
            1>  
          
          Show
          Dawid Weiss added a comment - Robert, this would mean it works fine, right (note dumped randomVal for each suite)? Executing 296 suites with 4 JVMs. Suite: org.apache.lucene.util.TestCloseableThreadLocal > (@BeforeClass output) 1> randomVal: 9 1> OK 0.05s J1 | TestCloseableThreadLocal.testDefaultValueWithoutSetting OK 0.01s J1 | TestCloseableThreadLocal.testInitValue OK 0.01s J1 | TestCloseableThreadLocal.testNullValue Completed on J1 in 0.27s, 3 tests Suite: org.apache.lucene.util.TestTwoPhaseCommitTool > (@BeforeClass output) 1> randomVal: 6 1> OK 0.04s J2 | TestTwoPhaseCommitTool.testRollback OK 0.01s J2 | TestTwoPhaseCommitTool.testNullTPCs OK 0.01s J2 | TestTwoPhaseCommitTool.testWrapper OK 0.01s J2 | TestTwoPhaseCommitTool.testPrepareThenCommit Completed on J2 in 0.37s, 4 tests Suite: org.apache.lucene.util.TestNamedSPILoader > (@BeforeClass output) 1> randomVal: 7 1> OK 0.04s J0 | TestNamedSPILoader.testAvailableServices OK 0.01s J0 | TestNamedSPILoader.testBogusLookup OK 0.01s J0 | TestNamedSPILoader.testLookup Completed on J0 in 0.34s, 3 tests Suite: org.apache.lucene.util.TestSmallFloat > (@BeforeClass output) 1> randomVal: 2 1> OK 0.20s J3 | TestSmallFloat.testFloatToByte OK 0.01s J3 | TestSmallFloat.testByteToFloat Completed on J3 in 0.48s, 2 tests Suite: org.apache.lucene.index.TestTerm > (@BeforeClass output) 1> randomVal: 0 1>
          Hide
          Robert Muir added a comment -

          yeah that looks right to me. I think currently randomVal will stay the same.

          Show
          Robert Muir added a comment - yeah that looks right to me. I think currently randomVal will stay the same.
          Hide
          Dawid Weiss added a comment -

          Fixed in trunk.

          Show
          Dawid Weiss added a comment - Fixed in trunk.
          Dawid Weiss made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Fix Version/s 4.0 [ 12314025 ]
          Resolution Fixed [ 1 ]
          Uwe Schindler made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Dawid Weiss
              Reporter:
              Robert Muir
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development