Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-6554

ToBlockJoinFieldComparator wrapping is illegal

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3
    • Component/s: None
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      The following test case triggers an AssertionError:

        public void testMissingValues() throws IOException {
          final Directory dir = newDirectory();
          final RandomIndexWriter w = new RandomIndexWriter(random(), dir, newIndexWriterConfig(new MockAnalyzer(random()))
              .setMergePolicy(NoMergePolicy.INSTANCE));
          w.addDocument(new Document());
          w.getReader().close();
          w.addDocument(new Document());
          IndexReader reader = w.getReader();
          w.close();
          IndexSearcher searcher = newSearcher(reader);
          // all docs are parent
          BitDocIdSetFilter parentFilter = new BitDocIdSetCachingWrapperFilter(new QueryWrapperFilter(new MatchAllDocsQuery()));
          BitDocIdSetFilter childFilter = new BitDocIdSetCachingWrapperFilter(new QueryWrapperFilter(new MatchNoDocsQuery()));
          ToParentBlockJoinSortField sortField = new ToParentBlockJoinSortField(
              "some_random_field", SortField.Type.STRING, false, parentFilter, childFilter
          );
          Sort sort = new Sort(sortField);
          TopFieldDocs topDocs = searcher.search(new MatchAllDocsQuery(), 1, sort);
          searcher.getIndexReader().close();
          dir.close();
        }
      
      java.lang.AssertionError
      	at __randomizedtesting.SeedInfo.seed([E9D45D81F597AE4B:83490FC7D11D9ABA]:0)
      	at org.apache.lucene.search.FieldComparator$TermOrdValComparator.setBottom(FieldComparator.java:800)
      	at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getLeafComparator(FieldComparator.java:783)
      	at org.apache.lucene.search.join.ToParentBlockJoinFieldComparator.doSetNextReader(ToParentBlockJoinFieldComparator.java:83)
      	at org.apache.lucene.search.SimpleFieldComparator.getLeafComparator(SimpleFieldComparator.java:36)
      	at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
      	at org.apache.lucene.search.TopFieldCollector$NonScoringCollector.getLeafCollector(TopFieldCollector.java:141)
      	at org.apache.lucene.search.FilterCollector.getLeafCollector(FilterCollector.java:40)
      	at org.apache.lucene.search.AssertingCollector.getLeafCollector(AssertingCollector.java:48)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:611)
      	at org.apache.lucene.search.AssertingIndexSearcher.search(AssertingIndexSearcher.java:92)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:424)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:543)
      	at org.apache.lucene.search.IndexSearcher.searchAfter(IndexSearcher.java:528)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:455)
      	at org.apache.lucene.search.join.TestBlockJoinSorting.testMissingValues(TestBlockJoinSorting.java:347)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:483)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1627)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:836)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:872)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:886)
      	at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:50)
      	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
      	at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:49)
      	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
      	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
      	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
      	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
      	at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:798)
      	at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:458)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:845)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$3.evaluate(RandomizedRunner.java:747)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$4.evaluate(RandomizedRunner.java:781)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:792)
      	at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:46)
      	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
      	at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:42)
      	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
      	at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:39)
      	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
      	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
      	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
      	at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:54)
      	at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:48)
      	at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:65)
      	at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:55)
      	at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
      	at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:365)
      	at java.lang.Thread.run(Thread.java:745)
      

      The reason is that when a parent document does not have children, ToParentBlockJoinComparator simply omits to forward calls to copy to the wrapped comparator. So the wrapped comparator ends up with allocated slots that have 0 as an ordinal (the default value in an array) and a null value, which is illegal since 0 is a legal ordinal which can't map to null.

      This can't be fixed without adding new methods to the already crazy comparator API, so I think there is nothing we can do but remove this comparator.

      It would be possible to achieve the same functionnality by implementing something similar to SortedNumericSelector, except that it would have to select across several docs instead of values.

        Attachments

        1. LUCENE-6554.patch
          32 kB
          Adrien Grand
        2. LUCENE-6554.patch
          35 kB
          Adrien Grand

          Activity

            People

            • Assignee:
              jpountz Adrien Grand
              Reporter:
              jpountz Adrien Grand
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: