Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      ant test -Dtestcase=TestPayloadNearQuery -Dtests.method=test -Dtests.seed=24743B1132665845 -Dtests.slow=true -Dtests.locale=es_NI -Dtests.timezone=Israel -Dtests.asserts=true -Dtests.file.encoding=US-ASCII

         [junit4] Started J0 PID(19895@localhost).
         [junit4] Suite: org.apache.lucene.search.payloads.TestPayloadNearQuery
         [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPayloadNearQuery -Dtests.method=test -Dtests.seed=24743B1132665845 -Dtests.slow=true -Dtests.locale=es_NI -Dtests.timezone=Israel -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
         [junit4] ERROR   0.09s | TestPayloadNearQuery.test <<<
         [junit4]    > Throwable #1: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.lang.NullPointerException
         [junit4]    > 	at __randomizedtesting.SeedInfo.seed([24743B1132665845:AC2004CB9C9A35BD]:0)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:353)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:382)
         [junit4]    > 	at org.apache.lucene.search.payloads.TestPayloadNearQuery.test(TestPayloadNearQuery.java:144)
         [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
         [junit4]    > Caused by: java.util.concurrent.ExecutionException: java.lang.NullPointerException
         [junit4]    > 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
         [junit4]    > 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:665)
         [junit4]    > 	... 39 more
         [junit4]    > Caused by: java.lang.NullPointerException
         [junit4]    > 	at org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:202)
         [junit4]    > 	at org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:223)
         [junit4]    > 	at org.apache.lucene.search.spans.SpanScorer.ensureFreq(SpanScorer.java:65)
         [junit4]    > 	at org.apache.lucene.search.spans.SpanScorer.score(SpanScorer.java:118)
         [junit4]    > 	at org.apache.lucene.search.AssertingScorer.score(AssertingScorer.java:67)
         [junit4]    > 	at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:64)
         [junit4]    > 	at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:53)
         [junit4]    > 	at org.apache.lucene.search.AssertingCollector$1.collect(AssertingCollector.java:57)
         [junit4]    > 	at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:53)
         [junit4]    > 	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:203)
         [junit4]    > 	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:174)
         [junit4]    > 	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
         [junit4]    > 	at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:69)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:714)
         [junit4]    > 	at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:93)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher$4.call(IndexSearcher.java:656)
         [junit4]    > 	at org.apache.lucene.search.IndexSearcher$4.call(IndexSearcher.java:653)
         [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:265)
         [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
         [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
         [junit4]    > 	... 1 more
         [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=25825, maxDocsPerChunk=709, blockSize=459), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=25825, blockSize=459)), sim=DefaultSimilarity, locale=es_NI, timezone=Israel
         [junit4]   2> NOTE: Linux 3.13.0-49-generic amd64/Oracle Corporation 1.9.0-ea (64-bit)/cpus=8,threads=1,free=199771008,total=253231104
         [junit4]   2> NOTE: All tests run in this JVM: [TestPayloadNearQuery]
         [junit4] Completed [1/1] in 0.71s, 1 test, 1 error <<< FAILURES!
      
      1. LUCENE-6490.patch
        19 kB
        Alan Woodward

        Activity

        Hide
        Robert Muir added a comment -

        Note that original jenkins failure was with ConcurrentModificationException:

        http://build-eu-00.elastic.co/job/lucene_linux_java8_64_test_only/47773/testReport/junit/org.apache.lucene.search.payloads/TestPayloadNearQuery/test/

         [junit4] Suite: org.apache.lucene.search.payloads.TestPayloadNearQuery
           [junit4]   2> NOTE: reproduce with: ant test  -Dtestcase=TestPayloadNearQuery -Dtests.method=test -Dtests.seed=24743B1132665845 -Dtests.slow=true -Dtests.locale=es_NI -Dtests.timezone=Israel -Dtests.asserts=true -Dtests.file.encoding=US-ASCII
           [junit4] ERROR   0.01s J5 | TestPayloadNearQuery.test <<<
           [junit4]    > Throwable #1: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
           [junit4]    > 	at __randomizedtesting.SeedInfo.seed([24743B1132665845:AC2004CB9C9A35BD]:0)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:353)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:382)
           [junit4]    > 	at org.apache.lucene.search.payloads.TestPayloadNearQuery.test(TestPayloadNearQuery.java:158)
           [junit4]    > 	at java.lang.Thread.run(Thread.java:745)
           [junit4]    > Caused by: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException
           [junit4]    > 	at java.util.concurrent.FutureTask.report(FutureTask.java:122)
           [junit4]    > 	at java.util.concurrent.FutureTask.get(FutureTask.java:192)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:665)
           [junit4]    > 	... 39 more
           [junit4]    > Caused by: java.util.ConcurrentModificationException
           [junit4]    > 	at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901)
           [junit4]    > 	at java.util.ArrayList$Itr.next(ArrayList.java:851)
           [junit4]    > 	at org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:199)
           [junit4]    > 	at org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:223)
           [junit4]    > 	at org.apache.lucene.search.spans.SpanScorer.ensureFreq(SpanScorer.java:65)
           [junit4]    > 	at org.apache.lucene.search.spans.SpanScorer.score(SpanScorer.java:118)
           [junit4]    > 	at org.apache.lucene.search.AssertingScorer.score(AssertingScorer.java:67)
           [junit4]    > 	at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:64)
           [junit4]    > 	at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:53)
           [junit4]    > 	at org.apache.lucene.search.AssertingCollector$1.collect(AssertingCollector.java:57)
           [junit4]    > 	at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:53)
           [junit4]    > 	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:203)
           [junit4]    > 	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:174)
           [junit4]    > 	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35)
           [junit4]    > 	at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:69)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:714)
           [junit4]    > 	at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:93)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher$4.call(IndexSearcher.java:656)
           [junit4]    > 	at org.apache.lucene.search.IndexSearcher$4.call(IndexSearcher.java:653)
           [junit4]    > 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
           [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
           [junit4]    > 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
           [junit4]    > 	... 1 more
           [junit4]   2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=25825, maxDocsPerChunk=709, blockSize=459), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=25825, blockSize=459)), sim=DefaultSimilarity, locale=es_NI, timezone=Israel
           [junit4]   2> NOTE: Linux 3.13.0-35-generic amd64/Oracle Corporation 1.8.0_40 (64-bit)/cpus=8,threads=1,free=306859232,total=519569408
           [junit4]   2> NOTE: All tests run in this JVM: [TestPerFieldPostingsFormat, TestTermRangeFilter, TestDocTermOrdsRangeFilter, TestSimilarityBase, TestFieldValueQuery, TestSimpleAttributeImpl, TestWindowsMMap, TestDateSort, TestCharsRef, Test2BNumericDocValues, TestTragicIndexWriterDeadlock, TestMinimize, TestFastDecompressionMode, TestFastCompressionMode, TestSpansAdvanced2, TestHighCompressionMode, TestLRUFilterCache, TestMultiDocValues, TestAtomicUpdate, TestMergeRateLimiter, TestPayloadNearQuery]
        
        Show
        Robert Muir added a comment - Note that original jenkins failure was with ConcurrentModificationException: http://build-eu-00.elastic.co/job/lucene_linux_java8_64_test_only/47773/testReport/junit/org.apache.lucene.search.payloads/TestPayloadNearQuery/test/ [junit4] Suite: org.apache.lucene.search.payloads.TestPayloadNearQuery [junit4] 2> NOTE: reproduce with: ant test -Dtestcase=TestPayloadNearQuery -Dtests.method=test -Dtests.seed=24743B1132665845 -Dtests.slow=true -Dtests.locale=es_NI -Dtests.timezone=Israel -Dtests.asserts=true -Dtests.file.encoding=US-ASCII [junit4] ERROR 0.01s J5 | TestPayloadNearQuery.test <<< [junit4] > Throwable #1: java.lang.RuntimeException: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException [junit4] > at __randomizedtesting.SeedInfo.seed([24743B1132665845:AC2004CB9C9A35BD]:0) [junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:669) [junit4] > at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:353) [junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:382) [junit4] > at org.apache.lucene.search.payloads.TestPayloadNearQuery.test(TestPayloadNearQuery.java:158) [junit4] > at java.lang.Thread.run(Thread.java:745) [junit4] > Caused by: java.util.concurrent.ExecutionException: java.util.ConcurrentModificationException [junit4] > at java.util.concurrent.FutureTask.report(FutureTask.java:122) [junit4] > at java.util.concurrent.FutureTask.get(FutureTask.java:192) [junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:665) [junit4] > ... 39 more [junit4] > Caused by: java.util.ConcurrentModificationException [junit4] > at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:901) [junit4] > at java.util.ArrayList$Itr.next(ArrayList.java:851) [junit4] > at org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.processPayloads(PayloadNearQuery.java:199) [junit4] > at org.apache.lucene.search.payloads.PayloadNearQuery$PayloadNearSpanScorer.setFreqCurrentDoc(PayloadNearQuery.java:223) [junit4] > at org.apache.lucene.search.spans.SpanScorer.ensureFreq(SpanScorer.java:65) [junit4] > at org.apache.lucene.search.spans.SpanScorer.score(SpanScorer.java:118) [junit4] > at org.apache.lucene.search.AssertingScorer.score(AssertingScorer.java:67) [junit4] > at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:64) [junit4] > at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:53) [junit4] > at org.apache.lucene.search.AssertingCollector$1.collect(AssertingCollector.java:57) [junit4] > at org.apache.lucene.search.AssertingLeafCollector.collect(AssertingLeafCollector.java:53) [junit4] > at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreAll(Weight.java:203) [junit4] > at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:174) [junit4] > at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:35) [junit4] > at org.apache.lucene.search.AssertingBulkScorer.score(AssertingBulkScorer.java:69) [junit4] > at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:714) [junit4] > at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:93) [junit4] > at org.apache.lucene.search.IndexSearcher$4.call(IndexSearcher.java:656) [junit4] > at org.apache.lucene.search.IndexSearcher$4.call(IndexSearcher.java:653) [junit4] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) [junit4] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [junit4] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [junit4] > ... 1 more [junit4] 2> NOTE: test params are: codec=FastDecompressionCompressingStoredFields(storedFieldsFormat=CompressingStoredFieldsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=25825, maxDocsPerChunk=709, blockSize=459), termVectorsFormat=CompressingTermVectorsFormat(compressionMode=FAST_DECOMPRESSION, chunkSize=25825, blockSize=459)), sim=DefaultSimilarity, locale=es_NI, timezone=Israel [junit4] 2> NOTE: Linux 3.13.0-35-generic amd64/Oracle Corporation 1.8.0_40 (64-bit)/cpus=8,threads=1,free=306859232,total=519569408 [junit4] 2> NOTE: All tests run in this JVM: [TestPerFieldPostingsFormat, TestTermRangeFilter, TestDocTermOrdsRangeFilter, TestSimilarityBase, TestFieldValueQuery, TestSimpleAttributeImpl, TestWindowsMMap, TestDateSort, TestCharsRef, Test2BNumericDocValues, TestTragicIndexWriterDeadlock, TestMinimize, TestFastDecompressionMode, TestFastCompressionMode, TestSpansAdvanced2, TestHighCompressionMode, TestLRUFilterCache, TestMultiDocValues, TestAtomicUpdate, TestMergeRateLimiter, TestPayloadNearQuery]
        Hide
        Alan Woodward added a comment -

        We're going to need a different SpanCollector per-scorer to deal with multithreaded search. Working on a patch now.

        Randomized testing ftw...

        Show
        Alan Woodward added a comment - We're going to need a different SpanCollector per-scorer to deal with multithreaded search. Working on a patch now. Randomized testing ftw...
        Hide
        Dawid Weiss added a comment -

        > Randomized testing ftw..

        https://goo.gl/kdp1dz

        Show
        Dawid Weiss added a comment - > Randomized testing ftw.. https://goo.gl/kdp1dz
        Hide
        Alan Woodward added a comment -

        This patch introduces a SpanCollectorFactory interface, and SpanWeight now takes this instead of SpanCollector, generating a new collector each time scorer() is called.

        Running tests and precommit now.

        Show
        Alan Woodward added a comment - This patch introduces a SpanCollectorFactory interface, and SpanWeight now takes this instead of SpanCollector, generating a new collector each time scorer() is called. Running tests and precommit now.
        Hide
        ASF subversion and git services added a comment -

        Commit 1680500 from Alan Woodward in branch 'dev/trunk'
        [ https://svn.apache.org/r1680500 ]

        LUCENE-6490: SpanCollector must be created per-scorer to allow multithreaded search

        Show
        ASF subversion and git services added a comment - Commit 1680500 from Alan Woodward in branch 'dev/trunk' [ https://svn.apache.org/r1680500 ] LUCENE-6490 : SpanCollector must be created per-scorer to allow multithreaded search
        Hide
        ASF subversion and git services added a comment -

        Commit 1680514 from Alan Woodward in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1680514 ]

        LUCENE-6490: SpanCollector must be created per-scorer to allow multithreaded search

        Show
        ASF subversion and git services added a comment - Commit 1680514 from Alan Woodward in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1680514 ] LUCENE-6490 : SpanCollector must be created per-scorer to allow multithreaded search
        Hide
        Anshum Gupta added a comment -

        I'll revert r1680514 from branch_5x and change the fix version on this to be 5.3.

        Show
        Anshum Gupta added a comment - I'll revert r1680514 from branch_5x and change the fix version on this to be 5.3.
        Hide
        Robert Muir added a comment -

        I marked this one 5.3, i dont think we need to reopen it, as there is already a better solution for this on LUCENE-6494

        Show
        Robert Muir added a comment - I marked this one 5.3, i dont think we need to reopen it, as there is already a better solution for this on LUCENE-6494
        Hide
        Shalin Shekhar Mangar added a comment -

        Bulk close for 5.3.0 release

        Show
        Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

          People

          • Assignee:
            Alan Woodward
            Reporter:
            Robert Muir
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development