Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-352

[PATCH] NullPointerException when using nested SpanOrQuery in SpanNotQuery

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.4
    • Fix Version/s: 2.0.0
    • Component/s: core/search
    • Labels:
      None
    • Environment:

      Operating System: Linux
      Platform: Other

    • Bugzilla Id:
      33725

      Description

      Overview description:
      I'm using the span query classes in Lucene to generate higher scores for
      search results where the search terms are closer together. In certain
      situations I want to exclude terms from the span. When I attempt to exclude
      more than one term I get an error.

      The example query I'm using is:

      'brighton AND tourism' -pier -contents

      I construct the query objects and the toString() version is:

      spanNot(spanNear([contents:brighton contents:tourism], 10, false),
      spanOr([contents:pier, contents:road]))

      Steps to reproduce:
      1. Construct a SpanNearQuery (must have at least one term, but at least two
      makes more sense)
      2. Construct a SpanOrQuery containing two or more terms
      3. Construct a SpanNotQuery to include the first query object and exclude the
      second (SpanOrQuery)
      4. Execute the search

      Actual Results:
      A null pointer exception is thrown while generating the scores within the
      search.

      Stack trace:
      java.lang.NullPointerException
      at
      org.apache.lucene.search.spans.SpanOrQuery$1.doc(SpanOrQuery.java:174)
      at
      org.apache.lucene.search.spans.SpanNotQuery$1.next(SpanNotQuery.java:75)
      at org.apache.lucene.search.spans.SpanScorer.next(SpanScorer.java:50)
      at org.apache.lucene.search.Scorer.score(Scorer.java:37)
      at
      org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:92)
      at org.apache.lucene.search.Hits.getMoreDocs(Hits.java:64)
      at org.apache.lucene.search.Hits.<init>(Hits.java:43)
      at org.apache.lucene.search.Searcher.search(Searcher.java:33)
      at org.apache.lucene.search.Searcher.search(Searcher.java:27)
      at
      com.runtimecollective.search.LuceneSearch.search(LuceneSearch.java:362)

      Expected Resuts:
      It executes the search and results where the first search terms (near query)
      are close together but without the second terms (or query) appearing.

        Issue Links

          Activity

          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          The bug could be in SpanNotQuery.java, line 67,
          which sets moreExclude to true.

          Could you try with:

          private boolean moreExclude = excludeSpans.next();

          instead?

          Regards,
          Paul Elschot

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - The bug could be in SpanNotQuery.java, line 67, which sets moreExclude to true. Could you try with: private boolean moreExclude = excludeSpans.next(); instead? Regards, Paul Elschot
          Hide
          miles@runtime-collective.com Miles Barr added a comment -

          Created an attachment (id=14356)
          Change suggested by Paul Elschot

          Changing moreExlude from 'true' to 'excludeSpans.next()' did the trick.

          Show
          miles@runtime-collective.com Miles Barr added a comment - Created an attachment (id=14356) Change suggested by Paul Elschot Changing moreExlude from 'true' to 'excludeSpans.next()' did the trick.
          Hide
          miles@runtime-collective.com Miles Barr added a comment -

          That should be 'moreExclude'.

          Show
          miles@runtime-collective.com Miles Barr added a comment - That should be 'moreExclude'.
          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          (In reply to comment #2)
          > Created an attachment (id=14356) [edit]
          > Change suggested by Paul Elschot
          >
          > Changing moreExlude from 'true' to 'excludeSpans.next()' did the trick.

          That's good news. Thanks for the patch.

          Would it be possible for you to add a few test cases
          for this?

          org.apache.lucune.search.spans.TestSpans in the src/test
          directory has some test examples.

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - (In reply to comment #2) > Created an attachment (id=14356) [edit] > Change suggested by Paul Elschot > > Changing moreExlude from 'true' to 'excludeSpans.next()' did the trick. That's good news. Thanks for the patch. Would it be possible for you to add a few test cases for this? org.apache.lucune.search.spans.TestSpans in the src/test directory has some test examples.
          Hide
          miles@runtime-collective.com Miles Barr added a comment -

          Created an attachment (id=14357)
          Additional test cases for when using a SpanOrQuery as the exclude option

          Here's a patch file for

          test/src/org/apache/lucene/search/spans/TestBasics.java

          I added two test cases:

          1. When there's only one SpanQuery in the SpanOrQuery
          2. When there's more than one SpanQuery in the SpanOrQuery

          There's also a commented out test for when you have an empty SpanOrQuery but
          that currently throws an exception (complains that fields don't have the same
          name) when you try to run it.

          I only used regular SpanTermQuery objects within the SpanOrQuery. I assume the
          regular SpanOrQuery tests verify the SpanOrQuery works correctly with the
          different span classes.

          Show
          miles@runtime-collective.com Miles Barr added a comment - Created an attachment (id=14357) Additional test cases for when using a SpanOrQuery as the exclude option Here's a patch file for test/src/org/apache/lucene/search/spans/TestBasics.java I added two test cases: 1. When there's only one SpanQuery in the SpanOrQuery 2. When there's more than one SpanQuery in the SpanOrQuery There's also a commented out test for when you have an empty SpanOrQuery but that currently throws an exception (complains that fields don't have the same name) when you try to run it. I only used regular SpanTermQuery objects within the SpanOrQuery. I assume the regular SpanOrQuery tests verify the SpanOrQuery works correctly with the different span classes.
          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          Miles,

          The commented test case in the patch to TestBasics passes an empty
          SpanOrQuery to a SpanNotQuery. That is probably not intended.

          Regards,
          Paul Elschot

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - Miles, The commented test case in the patch to TestBasics passes an empty SpanOrQuery to a SpanNotQuery. That is probably not intended. Regards, Paul Elschot
          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          (In reply to comment #6)
          > Miles,
          >
          > The commented test case in the patch to TestBasics passes an empty
          > SpanOrQuery to a SpanNotQuery. That is probably not intended.

          I just reread the previous post, and it is intended.
          However, I don't get the point of passing an empty SpanOrQuery.

          >
          > Regards,
          > Paul Elschot
          >

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - (In reply to comment #6) > Miles, > > The commented test case in the patch to TestBasics passes an empty > SpanOrQuery to a SpanNotQuery. That is probably not intended. I just reread the previous post, and it is intended. However, I don't get the point of passing an empty SpanOrQuery. > > Regards, > Paul Elschot >
          Hide
          miles@runtime-collective.com Miles Barr added a comment -

          My reasoning was that a poorly written parser might construct it with an empty
          SpanOrQuery and it should probably produce the same results as just the
          include query. But it would be incorrect to do so, so you'd expect an error.

          If you don't want to change the behaviour then just throw out the commented
          test case.

          Show
          miles@runtime-collective.com Miles Barr added a comment - My reasoning was that a poorly written parser might construct it with an empty SpanOrQuery and it should probably produce the same results as just the include query. But it would be incorrect to do so, so you'd expect an error. If you don't want to change the behaviour then just throw out the commented test case.
          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          Created an attachment (id=14558)
          Patch to TestBasics.java with the additional test cases except the discussed
          one.

          I think this settles the discussion,
          Regards,
          Paul Elschot

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - Created an attachment (id=14558) Patch to TestBasics.java with the additional test cases except the discussed one. I think this settles the discussion, Regards, Paul Elschot
          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          The patch of 24 Feb 2005 has not yet been applied.
          This bug might be reopened because of that, but that would be overdoing it.

          The extra test code in the patch of 24 March 2005 seems ok to me.

          Regards,
          Paul Elschot

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - The patch of 24 Feb 2005 has not yet been applied. This bug might be reopened because of that, but that would be overdoing it. The extra test code in the patch of 24 March 2005 seems ok to me. Regards, Paul Elschot
          Hide
          paul.elschot@xs4all.nl Paul Elschot added a comment -

          The fix in patch.txt is not in the trunk, and there was some recent discussion on
          what might be the same bug, so I'm reopening this.

          Show
          paul.elschot@xs4all.nl Paul Elschot added a comment - The fix in patch.txt is not in the trunk, and there was some recent discussion on what might be the same bug, so I'm reopening this.
          Hide
          hossman Hoss Man added a comment -

          consolidated patch containing...

          • additional test methods in TestBasicsPatch2.txt
          • additional test methods from LUCENE-560
          • one line change to from patch.txt that makes these tests pass
          Show
          hossman Hoss Man added a comment - consolidated patch containing... additional test methods in TestBasicsPatch2.txt additional test methods from LUCENE-560 one line change to from patch.txt that makes these tests pass
          Hide
          hossman Hoss Man added a comment -

          I think this bug is actualy simple enough that I can honestly say I understand it ... so unless anyone obejcts i'll commit "LUCENE-352-consolidated-SpanNot-NPE.patch" in a few days.

          Show
          hossman Hoss Man added a comment - I think this bug is actualy simple enough that I can honestly say I understand it ... so unless anyone obejcts i'll commit " LUCENE-352 -consolidated-SpanNot-NPE.patch" in a few days.
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          +1
          looks fine.

          Show
          yseeley@gmail.com Yonik Seeley added a comment - +1 looks fine.
          Hide
          hossman Hoss Man added a comment -

          commited patch

          Show
          hossman Hoss Man added a comment - commited patch

            People

            • Assignee:
              hossman Hoss Man
              Reporter:
              miles@runtime-collective.com Miles Barr
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development