Lucene - Core
  1. Lucene - Core
  2. LUCENE-3184

add LuceneTestCase.rarely()/LuceneTestCase.atLeast()

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      in LUCENE-3175, the tests were sped up a lot by using reasonable number of iterations normally, but cranking up for NIGHTLY.
      we also do crazy things more 'rarely' for normal builds (e.g. simpletext, payloads, crazy merge params, etc)
      also, we found some bugs by doing this, because in general our parameters are too fixed.

      however, it made the code look messy... I propose some new methods:
      instead of some crazy code in your test like:

      int numdocs = (TEST_NIGHTLY ? 1000 : 100) * RANDOM_MULTIPLIER;
      

      you use:

      int numdocs = atLeast(100);
      

      this will apply the multiplier, also factor in nightly, and finally add some random fudge... so e.g. in local runs its sometimes 127 docs, sometimes 113 docs, etc.

      additionally instead of code like:

      if ((TEST_NIGHTLY && random.nextBoolean()) || (random.nextInt(20) == 17)) {
      

      you do

      if (rarely()) {
      

      which applies NIGHTLY and also the multiplier (logarithmic growth).

        Activity

        Robert Muir created issue -
        Hide
        Robert Muir added a comment -

        here's a patch, also includes some random speedups to some of these tests.

        Show
        Robert Muir added a comment - here's a patch, also includes some random speedups to some of these tests.
        Robert Muir made changes -
        Field Original Value New Value
        Attachment LUCENE-3184.patch [ 12481854 ]
        Hide
        Michael McCandless added a comment -

        Looks great! I love the LTC.rarely, usually, atLeast methods

        Show
        Michael McCandless added a comment - Looks great! I love the LTC.rarely, usually, atLeast methods
        rmuir committed 1133599 (30 files)
        Reviews: none

        LUCENE-3184: add LuceneTestCase.rarely/LuceneTestCase.atLeast

        Lucene trunk
        rmuir committed 1133603 (21 files)
        Reviews: none

        LUCENE-3184: add LuceneTestCase.rarely/LuceneTestCase.atLeast

        Lucene branch_3x
        Hide
        Robert Muir added a comment -

        we can fix the remaining tests to use this stuff as we get around to them.

        Show
        Robert Muir added a comment - we can fix the remaining tests to use this stuff as we get around to them.
        Robert Muir made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Uwe Schindler added a comment -

        There are some problems now in nightly builds with OOMs (boith 3.x and trunk). We can do either:

        • reduce some nightly settings
        • raise -Xmx if nightly=true, maybe dependent on number of iterations, too?
        Show
        Uwe Schindler added a comment - There are some problems now in nightly builds with OOMs (boith 3.x and trunk). We can do either: reduce some nightly settings raise -Xmx if nightly=true, maybe dependent on number of iterations, too?
        Uwe Schindler made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        Uwe Schindler added a comment -

        All OOMs are related to FST usage (mostly the one with the big file and lots of iterations):

        • org.apache.lucene.util.fst.TestFSTs.testBigSet

        but also:

        • org.apache.lucene.index.TestPerFieldCodecSupport.testStressPerFieldCodec
        • org.apache.lucene.search.TestPhraseQuery.testRandomPhrases
        Show
        Uwe Schindler added a comment - All OOMs are related to FST usage (mostly the one with the big file and lots of iterations): org.apache.lucene.util.fst.TestFSTs.testBigSet but also: org.apache.lucene.index.TestPerFieldCodecSupport.testStressPerFieldCodec org.apache.lucene.search.TestPhraseQuery.testRandomPhrases
        Hide
        Uwe Schindler added a comment -

        I have also to note, that the tests now take horribly long in nightly. One build took 2 hrs and was finally failing at testBigSet. So it was not even running module/contrib tests, it took 2hrs for test-core in Lucene. Clover did also not run, but it does not use nightly or more than one iteration.

        Show
        Uwe Schindler added a comment - I have also to note, that the tests now take horribly long in nightly. One build took 2 hrs and was finally failing at testBigSet. So it was not even running module/contrib tests, it took 2hrs for test-core in Lucene. Clover did also not run, but it does not use nightly or more than one iteration.
        Robert Muir made changes -
        Fix Version/s 3.4 [ 12316675 ]
        Fix Version/s 3.3 [ 12316470 ]
        Michael McCandless made changes -
        Fix Version/s 3.5 [ 12317877 ]
        Fix Version/s 3.4 [ 12316675 ]
        Hide
        Robert Muir added a comment -

        I think uwe's reported problems are fixed

        Show
        Robert Muir added a comment - I think uwe's reported problems are fixed
        Robert Muir made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Uwe Schindler added a comment -

        Bulk close after release of 3.5

        Show
        Uwe Schindler added a comment - Bulk close after release of 3.5
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development