Lucene - Core
  1. Lucene - Core
  2. LUCENE-3184

add LuceneTestCase.rarely()/LuceneTestCase.atLeast()

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      in LUCENE-3175, the tests were sped up a lot by using reasonable number of iterations normally, but cranking up for NIGHTLY.
      we also do crazy things more 'rarely' for normal builds (e.g. simpletext, payloads, crazy merge params, etc)
      also, we found some bugs by doing this, because in general our parameters are too fixed.

      however, it made the code look messy... I propose some new methods:
      instead of some crazy code in your test like:

      int numdocs = (TEST_NIGHTLY ? 1000 : 100) * RANDOM_MULTIPLIER;
      

      you use:

      int numdocs = atLeast(100);
      

      this will apply the multiplier, also factor in nightly, and finally add some random fudge... so e.g. in local runs its sometimes 127 docs, sometimes 113 docs, etc.

      additionally instead of code like:

      if ((TEST_NIGHTLY && random.nextBoolean()) || (random.nextInt(20) == 17)) {
      

      you do

      if (rarely()) {
      

      which applies NIGHTLY and also the multiplier (logarithmic growth).

        Activity

        Hide
        Robert Muir added a comment -

        here's a patch, also includes some random speedups to some of these tests.

        Show
        Robert Muir added a comment - here's a patch, also includes some random speedups to some of these tests.
        Hide
        Michael McCandless added a comment -

        Looks great! I love the LTC.rarely, usually, atLeast methods

        Show
        Michael McCandless added a comment - Looks great! I love the LTC.rarely, usually, atLeast methods
        Hide
        Robert Muir added a comment -

        we can fix the remaining tests to use this stuff as we get around to them.

        Show
        Robert Muir added a comment - we can fix the remaining tests to use this stuff as we get around to them.
        Hide
        Uwe Schindler added a comment -

        There are some problems now in nightly builds with OOMs (boith 3.x and trunk). We can do either:

        • reduce some nightly settings
        • raise -Xmx if nightly=true, maybe dependent on number of iterations, too?
        Show
        Uwe Schindler added a comment - There are some problems now in nightly builds with OOMs (boith 3.x and trunk). We can do either: reduce some nightly settings raise -Xmx if nightly=true, maybe dependent on number of iterations, too?
        Hide
        Uwe Schindler added a comment -

        All OOMs are related to FST usage (mostly the one with the big file and lots of iterations):

        • org.apache.lucene.util.fst.TestFSTs.testBigSet

        but also:

        • org.apache.lucene.index.TestPerFieldCodecSupport.testStressPerFieldCodec
        • org.apache.lucene.search.TestPhraseQuery.testRandomPhrases
        Show
        Uwe Schindler added a comment - All OOMs are related to FST usage (mostly the one with the big file and lots of iterations): org.apache.lucene.util.fst.TestFSTs.testBigSet but also: org.apache.lucene.index.TestPerFieldCodecSupport.testStressPerFieldCodec org.apache.lucene.search.TestPhraseQuery.testRandomPhrases
        Hide
        Uwe Schindler added a comment -

        I have also to note, that the tests now take horribly long in nightly. One build took 2 hrs and was finally failing at testBigSet. So it was not even running module/contrib tests, it took 2hrs for test-core in Lucene. Clover did also not run, but it does not use nightly or more than one iteration.

        Show
        Uwe Schindler added a comment - I have also to note, that the tests now take horribly long in nightly. One build took 2 hrs and was finally failing at testBigSet. So it was not even running module/contrib tests, it took 2hrs for test-core in Lucene. Clover did also not run, but it does not use nightly or more than one iteration.
        Hide
        Robert Muir added a comment -

        I think uwe's reported problems are fixed

        Show
        Robert Muir added a comment - I think uwe's reported problems are fixed
        Hide
        Uwe Schindler added a comment -

        Bulk close after release of 3.5

        Show
        Uwe Schindler added a comment - Bulk close after release of 3.5

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development