Lucene - Core
  1. Lucene - Core
  2. LUCENE-5450

NPE and Illegal Argument Exception in wrapped SpanMultiTerms with no matches

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 6.0
    • Fix Version/s: 4.7.1, 4.8, 6.0
    • Component/s: core/search
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      There are problems with the handling of wrapped span multiterms that don't have any matches.

      1) In the test framework, when AssertingIndexSearcher does a rewrite and then checks for equality, there's an NPE for SpanNear and SpanOr:

      java.lang.NullPointerException
      	at __randomizedtesting.SeedInfo.seed([8E96A398C89A703B:338EB53A2EDBE8CC]:0)
      	at org.apache.lucene.search.spans.SpanOrQuery.addClause(SpanOrQuery.java:57)
      	at org.apache.lucene.search.spans.SpanOrQuery.<init>(SpanOrQuery.java:49)
      	at org.apache.lucene.search.spans.SpanOrQuery.clone(SpanOrQuery.java:86)
      	at org.apache.lucene.search.spans.SpanOrQuery.clone(SpanOrQuery.java:39)
      	at org.apache.lucene.search.QueryUtils.checkHashEquals(QueryUtils.java:57)
      	at org.apache.lucene.search.QueryUtils.check(QueryUtils.java:52)
      	at org.apache.lucene.search.AssertingIndexSearcher.rewrite(AssertingIndexSearcher.java:80)
      	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:675)
      	at org.apache.lucene.search.AssertingIndexSearcher.createNormalizedWeight(AssertingIndexSearcher.java:59)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:273)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:261)
      	at org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper.testNoSuchMultiTermsInOr(TestSpanMultiTermQueryWrapper.java:177)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
      	at java.lang.reflect.Method.invoke(Unknown Source)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1559)
      ...
      
      

      2) The same issue is causing a "Clauses must have same field" illegal argument exception in a SpanNotQuery.

      java.lang.IllegalArgumentException: Clauses must have same field.
      	at __randomizedtesting.SeedInfo.seed([779E5DD7E7523C72:4C2ECAAB938038F9]:0)
      	at org.apache.lucene.search.spans.SpanNotQuery.<init>(SpanNotQuery.java:66)
      	at org.apache.lucene.search.spans.SpanNotQuery.clone(SpanNotQuery.java:99)
      	at org.apache.lucene.search.spans.SpanNotQuery.clone(SpanNotQuery.java:36)
      	at org.apache.lucene.search.QueryUtils.checkHashEquals(QueryUtils.java:57)
      	at org.apache.lucene.search.QueryUtils.check(QueryUtils.java:52)
      	at org.apache.lucene.search.AssertingIndexSearcher.rewrite(AssertingIndexSearcher.java:80)
      	at org.apache.lucene.search.IndexSearcher.createNormalizedWeight(IndexSearcher.java:675)
      	at org.apache.lucene.search.AssertingIndexSearcher.createNormalizedWeight(AssertingIndexSearcher.java:59)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:273)
      	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:261)
      	at org.apache.lucene.search.spans.TestSpanMultiTermQueryWrapper.testNoSuchMultiTermsInNotNear(TestSpanMultiTermQueryWrapper.java:144)
      ...
      

      The basic problem is that an empty SpanQuery (SpanOrQuery with zero clauses) does not have a field, and much of our code assumes that the field is not null.

      Test case and patch on way.

      1. LUCENE-5450.patch
        11 kB
        Robert Muir
      2. LUCENE-5450.patch
        13 kB
        Tim Allison

        Activity

        Hide
        Tim Allison added a comment -

        Test case and v1 of patch attached.

        Show
        Tim Allison added a comment - Test case and v1 of patch attached.
        Hide
        Robert Muir added a comment -

        Hi Tim, thanks for reporting the bug.

        Personally, i don't like the use of the static isEmpty method with instanceofs, because it makes it harder for people to implement their own queries.
        An alternative solution (that is no change in behavior and no api change today, since getField() is currently null in those cases) is to just add the proper null checks?

        Show
        Robert Muir added a comment - Hi Tim, thanks for reporting the bug. Personally, i don't like the use of the static isEmpty method with instanceofs, because it makes it harder for people to implement their own queries. An alternative solution (that is no change in behavior and no api change today, since getField() is currently null in those cases) is to just add the proper null checks?
        Hide
        Robert Muir added a comment -

        Here's what i was thinking (your tests and existing tests seem to pass). Let me know what you think.

        Show
        Robert Muir added a comment - Here's what i was thinking (your tests and existing tests seem to pass). Let me know what you think.
        Hide
        Tim Allison added a comment -

        Looks great. Much simpler. Thank you!

        Show
        Tim Allison added a comment - Looks great. Much simpler. Thank you!
        Hide
        Michael McCandless added a comment -

        +1

        Show
        Michael McCandless added a comment - +1
        Hide
        ASF subversion and git services added a comment -

        Commit 1569503 from Robert Muir in branch 'dev/trunk'
        [ https://svn.apache.org/r1569503 ]

        LUCENE-5450: fix getField() NPE issues with span queries that have empty clauses

        Show
        ASF subversion and git services added a comment - Commit 1569503 from Robert Muir in branch 'dev/trunk' [ https://svn.apache.org/r1569503 ] LUCENE-5450 : fix getField() NPE issues with span queries that have empty clauses
        Hide
        ASF subversion and git services added a comment -

        Commit 1569513 from Robert Muir in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1569513 ]

        LUCENE-5450: fix getField() NPE issues with span queries that have empty clauses

        Show
        ASF subversion and git services added a comment - Commit 1569513 from Robert Muir in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1569513 ] LUCENE-5450 : fix getField() NPE issues with span queries that have empty clauses
        Hide
        Robert Muir added a comment -

        Thanks Tim!

        Show
        Robert Muir added a comment - Thanks Tim!
        Hide
        Robert Muir added a comment -

        Reopening for 4.7.1 backport

        Show
        Robert Muir added a comment - Reopening for 4.7.1 backport
        Hide
        ASF subversion and git services added a comment -

        Commit 1578513 from Robert Muir in branch 'dev/branches/lucene_solr_4_7'
        [ https://svn.apache.org/r1578513 ]

        LUCENE-5450: NPE and Illegal Argument Exception in wrapped SpanMultiTerms with no matches

        Show
        ASF subversion and git services added a comment - Commit 1578513 from Robert Muir in branch 'dev/branches/lucene_solr_4_7' [ https://svn.apache.org/r1578513 ] LUCENE-5450 : NPE and Illegal Argument Exception in wrapped SpanMultiTerms with no matches
        Hide
        Steve Rowe added a comment -

        Bulk close 4.7.1 issues

        Show
        Steve Rowe added a comment - Bulk close 4.7.1 issues

          People

          • Assignee:
            Unassigned
            Reporter:
            Tim Allison
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development