Lucene - Core
  1. Lucene - Core
  2. LUCENE-3184

add LuceneTestCase.rarely()/LuceneTestCase.atLeast()

    Details

    • Type: Test Test
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5, 4.0-ALPHA
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New, Patch Available

      Description

      in LUCENE-3175, the tests were sped up a lot by using reasonable number of iterations normally, but cranking up for NIGHTLY.
      we also do crazy things more 'rarely' for normal builds (e.g. simpletext, payloads, crazy merge params, etc)
      also, we found some bugs by doing this, because in general our parameters are too fixed.

      however, it made the code look messy... I propose some new methods:
      instead of some crazy code in your test like:

      int numdocs = (TEST_NIGHTLY ? 1000 : 100) * RANDOM_MULTIPLIER;
      

      you use:

      int numdocs = atLeast(100);
      

      this will apply the multiplier, also factor in nightly, and finally add some random fudge... so e.g. in local runs its sometimes 127 docs, sometimes 113 docs, etc.

      additionally instead of code like:

      if ((TEST_NIGHTLY && random.nextBoolean()) || (random.nextInt(20) == 17)) {
      

      you do

      if (rarely()) {
      

      which applies NIGHTLY and also the multiplier (logarithmic growth).

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Resolved Resolved
        3h 49m 1 Robert Muir 09/Jun/11 00:40
        Resolved Resolved Reopened Reopened
        4d 11h 33m 1 Uwe Schindler 13/Jun/11 12:13
        Reopened Reopened Resolved Resolved
        138d 17h 17m 1 Robert Muir 30/Oct/11 04:31
        Resolved Resolved Closed Closed
        28d 7h 57m 1 Uwe Schindler 27/Nov/11 12:29
        Uwe Schindler made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Hide
        Uwe Schindler added a comment -

        Bulk close after release of 3.5

        Show
        Uwe Schindler added a comment - Bulk close after release of 3.5
        Robert Muir made changes -
        Status Reopened [ 4 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Robert Muir added a comment -

        I think uwe's reported problems are fixed

        Show
        Robert Muir added a comment - I think uwe's reported problems are fixed
        Michael McCandless made changes -
        Fix Version/s 3.5 [ 12317877 ]
        Fix Version/s 3.4 [ 12316675 ]
        Robert Muir made changes -
        Fix Version/s 3.4 [ 12316675 ]
        Fix Version/s 3.3 [ 12316470 ]
        Hide
        Uwe Schindler added a comment -

        I have also to note, that the tests now take horribly long in nightly. One build took 2 hrs and was finally failing at testBigSet. So it was not even running module/contrib tests, it took 2hrs for test-core in Lucene. Clover did also not run, but it does not use nightly or more than one iteration.

        Show
        Uwe Schindler added a comment - I have also to note, that the tests now take horribly long in nightly. One build took 2 hrs and was finally failing at testBigSet. So it was not even running module/contrib tests, it took 2hrs for test-core in Lucene. Clover did also not run, but it does not use nightly or more than one iteration.
        Hide
        Uwe Schindler added a comment -

        All OOMs are related to FST usage (mostly the one with the big file and lots of iterations):

        • org.apache.lucene.util.fst.TestFSTs.testBigSet

        but also:

        • org.apache.lucene.index.TestPerFieldCodecSupport.testStressPerFieldCodec
        • org.apache.lucene.search.TestPhraseQuery.testRandomPhrases
        Show
        Uwe Schindler added a comment - All OOMs are related to FST usage (mostly the one with the big file and lots of iterations): org.apache.lucene.util.fst.TestFSTs.testBigSet but also: org.apache.lucene.index.TestPerFieldCodecSupport.testStressPerFieldCodec org.apache.lucene.search.TestPhraseQuery.testRandomPhrases
        Uwe Schindler made changes -
        Resolution Fixed [ 1 ]
        Status Resolved [ 5 ] Reopened [ 4 ]
        Hide
        Uwe Schindler added a comment -

        There are some problems now in nightly builds with OOMs (boith 3.x and trunk). We can do either:

        • reduce some nightly settings
        • raise -Xmx if nightly=true, maybe dependent on number of iterations, too?
        Show
        Uwe Schindler added a comment - There are some problems now in nightly builds with OOMs (boith 3.x and trunk). We can do either: reduce some nightly settings raise -Xmx if nightly=true, maybe dependent on number of iterations, too?
        Robert Muir made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]
        Hide
        Robert Muir added a comment -

        we can fix the remaining tests to use this stuff as we get around to them.

        Show
        Robert Muir added a comment - we can fix the remaining tests to use this stuff as we get around to them.
        Hide
        Michael McCandless added a comment -

        Looks great! I love the LTC.rarely, usually, atLeast methods

        Show
        Michael McCandless added a comment - Looks great! I love the LTC.rarely, usually, atLeast methods
        Robert Muir made changes -
        Field Original Value New Value
        Attachment LUCENE-3184.patch [ 12481854 ]
        Hide
        Robert Muir added a comment -

        here's a patch, also includes some random speedups to some of these tests.

        Show
        Robert Muir added a comment - here's a patch, also includes some random speedups to some of these tests.
        Robert Muir created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development