Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-8506

TestAddIndexes.testAddIndicesWithSoftDeletes() failures

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core/index, general/test
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      TestAddIndexes.testAddIndicesWithSoftDeletes() has failed a few times on Jenkins (see below), but the seeds don't reproduce 100%: of the two I tested, one didn't reproduce at all in 5 iterations, and the other reproduced 1/5 iterations.

      I beasted the suite on master 50 times using Miller's beasting script and it failed just once:

      ({{git show}} reports: {{commit 895bff46b2c3d3d027e693e11a2feac9f518191d}})
         [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAddIndexes -Dtests.method=testAddIndicesWithSoftDeletes -Dtests.seed=70F06A77576A75 -Dtests.slow=true -Dtests.badapples=true -Dtests.locale=ar-YE -Dtests.timezone=Iceland -Dtests.asserts=true -Dtests.file.encoding=UTF-8
         [junit4] ERROR   0.40s | TestAddIndexes.testAddIndicesWithSoftDeletes <<<
         [junit4]    > Throwable #1: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
         [junit4]    >        at __randomizedtesting.SeedInfo.seed([70F06A77576A75:2C240F2EC2524E6D]:0)
         [junit4]    >        at java.util.ArrayList.rangeCheck(ArrayList.java:657)
         [junit4]    >        at java.util.ArrayList.get(ArrayList.java:433)
         [junit4]    >        at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
         [junit4]    >        at org.apache.lucene.index.TestAddIndexes.testAddIndicesWithSoftDeletes(TestAddIndexes.java:1452)
         [junit4]    >        at java.lang.Thread.run(Thread.java:748)
      [...]
         [junit4]   2> NOTE: test params are: codec=Asserting(Lucene80): {c=BlockTreeOrds(blocksize=128), id=BlockTreeOrds(blocksize=128), f1=BlockTreeOrds(blocksize=128), f2=Lucene50(blocksize=128), version=Lucene50(blocksize=128), content=FSTOrd50}, docValues:{dv=DocValuesFormat(name=Asserting), soft_delete=DocValuesFormat(name=Lucene70), doc=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Direct), content=DocValuesFormat(name=Lucene70), doc2d=DocValuesFormat(name=Asserting)}, maxPointsInLeafNode=69, maxMBSortInHeap=5.930450658985216, sim=Asserting(org.apache.lucene.search.similarities.AssertingSimilarity@35111f6), locale=ar-YE, timezone=Iceland
         [junit4]   2> NOTE: Linux 4.1.0-custom2-amd64 amd64/Oracle Corporation 1.8.0_171 (64-bit)/cpus=16,threads=1,free=490798376,total=511180800
      

      Jenkins failures (all on branch_7x):

      From https://jenkins.thetaphi.de/job/Lucene-Solr-7.x-Solaris/814/:

      Checking out Revision 0789a77c2590f716fc3cedb247309068c3fc5d85 (refs/remotes/origin/branch_7x)
      [...]
        [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAddIndexes -Dtests.method=testAddIndicesWithSoftDeletes -Dtests.seed=B80C15673B91141 -Dtests.slow=true -Dtests.locale=es-SV -Dtests.timezone=Asia/Jakarta -Dtests.asserts=true -Dtests.file.encoding=UTF-8
        [junit4] ERROR   0.02s J0 | TestAddIndexes.testAddIndicesWithSoftDeletes <<<
        [junit4]    > Throwable #1: java.lang.IndexOutOfBoundsException: Index: 2, Size: 2
        [junit4]    > 	at __randomizedtesting.SeedInfo.seed([B80C15673B91141:27D43E12C6BC3559]:0)
        [junit4]    > 	at java.util.ArrayList.rangeCheck(ArrayList.java:657)
        [junit4]    > 	at java.util.ArrayList.get(ArrayList.java:433)
        [junit4]    > 	at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
        [junit4]    > 	at org.apache.lucene.index.TestAddIndexes.testAddIndicesWithSoftDeletes(TestAddIndexes.java:1455)
        [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
      [...]
        [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {c=PostingsFormat(name=LuceneVarGapDocFreqInterval), id=PostingsFormat(name=LuceneVarGapDocFreqInterval), f1=PostingsFormat(name=LuceneVarGapDocFreqInterval), f2=FSTOrd50, version=FSTOrd50, content=BlockTreeOrds(blocksize=128)}, docValues:{dv=DocValuesFormat(name=Direct), soft_delete=DocValuesFormat(name=Asserting), doc=DocValuesFormat(name=Asserting), id=DocValuesFormat(name=Lucene70), content=DocValuesFormat(name=Memory), doc2d=DocValuesFormat(name=Direct)}, maxPointsInLeafNode=584, maxMBSortInHeap=7.761726159162388, sim=RandomSimilarity(queryNorm=true): {foo=DFR I(ne)BZ(0.3), id=DFR GL2, content=IB SPL-L1}, locale=es-SV, timezone=Asia/Jakarta
        [junit4]   2> NOTE: SunOS 5.11 amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=3,threads=1,free=350279888,total=518979584
      

      From https://builds.apache.org/job/Lucene-Solr-SmokeRelease-7.x/317/:

      Checking out Revision ae6ceb5cb302b6db17637b5097f5f07740287fb3 (refs/remotes/origin/branch_7x)
      [...]
         [smoker]    [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAddIndexes -Dtests.method=testAddIndicesWithSoftDeletes -Dtests.seed=4B546299EE94E20A -Dtests.multiplier=2 -Dtests.locale=ar-KW -Dtests.timezone=America/Detroit -Dtests.asserts=true -Dtests.file.encoding=UTF-8
         [smoker]    [junit4] ERROR   0.10s J1 | TestAddIndexes.testAddIndicesWithSoftDeletes <<<
         [smoker]    [junit4]    > Throwable #1: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
         [smoker]    [junit4]    > 	at __randomizedtesting.SeedInfo.seed([4B546299EE94E20A:67009DDD5B91C612]:0)
         [smoker]    [junit4]    > 	at java.util.ArrayList.rangeCheck(ArrayList.java:657)
         [smoker]    [junit4]    > 	at java.util.ArrayList.get(ArrayList.java:433)
         [smoker]    [junit4]    > 	at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
         [smoker]    [junit4]    > 	at org.apache.lucene.index.TestAddIndexes.testAddIndicesWithSoftDeletes(TestAddIndexes.java:1455)
         [smoker]    [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
         [smoker]    [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {c=FST50, id=FST50, f1=FST50, f2=PostingsFormat(name=LuceneVarGapDocFreqInterval), version=PostingsFormat(name=LuceneVarGapDocFreqInterval), content=PostingsFormat(name=Memory)}, docValues:{dv=DocValuesFormat(name=Memory), soft_delete=DocValuesFormat(name=Direct), doc=DocValuesFormat(name=Direct), id=DocValuesFormat(name=Lucene70), content=DocValuesFormat(name=Lucene70), doc2d=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1189, maxMBSortInHeap=6.69463932745071, sim=RandomSimilarity(queryNorm=true): {foo=DFR I(ne)LZ(0.3), id=DFR I(F)B1, content=DFR I(n)L1}, locale=ar-KW, timezone=America/Detroit
         [smoker]    [junit4]   2> NOTE: Linux 4.4.0-130-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=208806664,total=283639808
      

      From https://builds.apache.org/job/Lucene-Solr-NightlyTests-7.x/321/:

      Checking out Revision 20123377479bd3d0689f106ec53aa9e1148a258a (refs/remotes/origin/branch_7x)
      [...]
         [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestAddIndexes -Dtests.method=testAddIndicesWithSoftDeletes -Dtests.seed=3E6C2EF5518B4BE1 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/home/jenkins/jenkins-slave/workspace/Lucene-Solr-NightlyTests-7.x/test-data/enwiki.random.lines.txt -Dtests.locale=da -Dtests.timezone=Asia/Tashkent -Dtests.asserts=true -Dtests.file.encoding=UTF-8
         [junit4] ERROR   0.15s J2 | TestAddIndexes.testAddIndicesWithSoftDeletes <<<
         [junit4]    > Throwable #1: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
         [junit4]    > 	at __randomizedtesting.SeedInfo.seed([3E6C2EF5518B4BE1:1238D1B1E48E6FF9]:0)
         [junit4]    > 	at java.util.ArrayList.rangeCheck(ArrayList.java:657)
         [junit4]    > 	at java.util.ArrayList.get(ArrayList.java:433)
         [junit4]    > 	at java.util.Collections$UnmodifiableList.get(Collections.java:1309)
         [junit4]    > 	at org.apache.lucene.index.TestAddIndexes.testAddIndicesWithSoftDeletes(TestAddIndexes.java:1455)
         [junit4]    > 	at java.lang.Thread.run(Thread.java:748)
      [...]
         [junit4]   2> NOTE: test params are: codec=Asserting(Lucene70): {c=PostingsFormat(name=Asserting), id=PostingsFormat(name=Asserting), f1=PostingsFormat(name=Asserting), f2=PostingsFormat(name=Memory), version=PostingsFormat(name=Memory), content=PostingsFormat(name=LuceneVarGapFixedInterval)}, docValues:{dv=DocValuesFormat(name=Memory), soft_delete=DocValuesFormat(name=Lucene70), doc=DocValuesFormat(name=Lucene70), id=DocValuesFormat(name=Lucene70), content=DocValuesFormat(name=Asserting), doc2d=DocValuesFormat(name=Memory)}, maxPointsInLeafNode=1878, maxMBSortInHeap=6.026209004751744, sim=RandomSimilarity(queryNorm=true): {foo=DFR I(F)Z(0.3), id=DFR I(F)1, content=IB SPL-L3(800.0)}, locale=da, timezone=Asia/Tashkent
         [junit4]   2> NOTE: Linux 4.4.0-130-generic amd64/Oracle Corporation 1.8.0_172 (64-bit)/cpus=4,threads=1,free=117255208,total=499646464
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                sarowe Steven Rowe
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: