Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9235

NPE when using non-numeric range query in deleteByQuery

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.0, 6.1
    • Fix Version/s: 6.2
    • Component/s: query parsers
    • Labels:
      None

      Description

      Upgrading from Solr 4.0.0 to 6.0.1/6.1.0, this old query suddenly got our indexing stuck:

      <delete><query>lastdate_a:{* TO 20160620} AND lastdate_p:{* TO 20160620} AND country:9</query></delete>
      

      with this error logged:

      2016-06-20 02:20:36.429 ERROR (commitScheduler-15-thread-1) [   x:mycore] o.a.s.u.CommitTracker auto commit error...:java.lang.NullPointerException
              at org.apache.solr.query.SolrRangeQuery.createDocSet(SolrRangeQuery.java:156)
              at org.apache.solr.query.SolrRangeQuery.access$200(SolrRangeQuery.java:57)
              at org.apache.solr.query.SolrRangeQuery$ConstWeight.getSegState(SolrRangeQuery.java:412)
              at org.apache.solr.query.SolrRangeQuery$ConstWeight.scorer(SolrRangeQuery.java:484)
              at org.apache.lucene.search.LRUQueryCache$CachingWrapperWeight.scorer(LRUQueryCache.java:617)
              at org.apache.lucene.search.BooleanWeight.scorer(BooleanWeight.java:389)
              at org.apache.solr.update.DeleteByQueryWrapper$1.scorer(DeleteByQueryWrapper.java:89)
              at org.apache.lucene.index.BufferedUpdatesStream.applyQueryDeletes(BufferedUpdatesStream.java:694)
              at org.apache.lucene.index.BufferedUpdatesStream.applyDeletesAndUpdates(BufferedUpdatesStream.java:262)
              at org.apache.lucene.index.IndexWriter.applyAllDeletesAndUpdates(IndexWriter.java:3187)
              at org.apache.lucene.index.IndexWriter.maybeApplyDeletes(IndexWriter.java:3173)
              at org.apache.lucene.index.IndexWriter.prepareCommitInternal(IndexWriter.java:2825)
              at org.apache.lucene.index.IndexWriter.commitInternal(IndexWriter.java:2989)
              at org.apache.lucene.index.IndexWriter.commit(IndexWriter.java:2956)
              at org.apache.solr.update.DirectUpdateHandler2.commit(DirectUpdateHandler2.java:619)
              at org.apache.solr.update.CommitTracker.run(CommitTracker.java:217)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
              at java.util.concurrent.FutureTask.run(FutureTask.java:266)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
              at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      

      The types were:

        <fieldType name="string" class="solr.StrField" sortMissingLast="true" omitNorms="true"/>
        <field name="lastdate_a" type="string" indexed="true" stored="false" />
        <field name="lastdate_p" type="string" indexed="true" stored="false" />
        <field name="country" type="string" indexed="true" stored="true" multiValued="true" />
      

      but changing the date fields into "integer" seems to avoid the problem:

        <fieldType name="integer" class="solr.TrieIntField" omitNorms="true"/>
      

        Issue Links

          Activity

          Hide
          andersm Anders Melchiorsen added a comment -

          I forgot to mention that the exception is from version 6.0.1. I did test with 6.1.0 and saw the same issue.

          Show
          andersm Anders Melchiorsen added a comment - I forgot to mention that the exception is from version 6.0.1. I did test with 6.1.0 and saw the same issue.
          Hide
          mkhludnev Mikhail Khludnev added a comment -

          my gut feeling, you can only use ranges in delete query with 5.5.x and earlier.

          Show
          mkhludnev Mikhail Khludnev added a comment - my gut feeling, you can only use ranges in delete query with 5.5.x and earlier.
          Hide
          andersm Anders Melchiorsen added a comment -

          You are obviously right, I cannot use it when it breaks like that. But are you suggesting that the feature has been removed intentionally? Even if ranges are indeed unsupported, crashing the core is a bit of a harsh way to report that.

          As I said, it still works for integers – so my problem is solved, though the bug is not.

          Show
          andersm Anders Melchiorsen added a comment - You are obviously right, I cannot use it when it breaks like that. But are you suggesting that the feature has been removed intentionally? Even if ranges are indeed unsupported, crashing the core is a bit of a harsh way to report that. As I said, it still works for integers – so my problem is solved, though the bug is not.
          Hide
          hossman Hoss Man added a comment -

          an NPE would never be intentional - Mikhail is just pointing out when he thiks it broke, and from what i can tell he's right.

          Attached patch beefs up TestRangeQuery to demonstrate this bug (and in general to better exercise more code paths in SolrRangeQuery which can vary based on whether the number of terms is mall enough to re-write to a BooleanQuery or not).

          The added tests can ail very easily demonstrating the same problem reported here, and i added (what seems to me like) a trivial fix - if we don't have a SolrIndexSearcher (which we don't when IndexWriter processes a DBQ) then completley bypass all the "doCheck" logic and fall through the the more classic TermEnum + DocIdSetBuilder iteration.

          Yonik Seeley - does this look correct to you?

          Show
          hossman Hoss Man added a comment - an NPE would never be intentional - Mikhail is just pointing out when he thiks it broke, and from what i can tell he's right. Attached patch beefs up TestRangeQuery to demonstrate this bug (and in general to better exercise more code paths in SolrRangeQuery which can vary based on whether the number of terms is mall enough to re-write to a BooleanQuery or not). The added tests can ail very easily demonstrating the same problem reported here, and i added (what seems to me like) a trivial fix - if we don't have a SolrIndexSearcher (which we don't when IndexWriter processes a DBQ) then completley bypass all the "doCheck" logic and fall through the the more classic TermEnum + DocIdSetBuilder iteration. Yonik Seeley - does this look correct to you?
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          Yeah, looks good!

          Show
          yseeley@gmail.com Yonik Seeley added a comment - Yeah, looks good!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 5abdcb8fa5d7263c45159eef42ec4f995d45728c in lucene-solr's branch refs/heads/branch_6x from Chris Hostetter
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5abdcb8 ]

          SOLR-9235: Fixed NPE when using non-numeric range query in deleteByQuery

          (cherry picked from commit 54b3945572acc9fd155740d5e6f628bfb2b3848f)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 5abdcb8fa5d7263c45159eef42ec4f995d45728c in lucene-solr's branch refs/heads/branch_6x from Chris Hostetter [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5abdcb8 ] SOLR-9235 : Fixed NPE when using non-numeric range query in deleteByQuery (cherry picked from commit 54b3945572acc9fd155740d5e6f628bfb2b3848f)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 54b3945572acc9fd155740d5e6f628bfb2b3848f in lucene-solr's branch refs/heads/master from Chris Hostetter
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=54b3945 ]

          SOLR-9235: Fixed NPE when using non-numeric range query in deleteByQuery

          Show
          jira-bot ASF subversion and git services added a comment - Commit 54b3945572acc9fd155740d5e6f628bfb2b3848f in lucene-solr's branch refs/heads/master from Chris Hostetter [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=54b3945 ] SOLR-9235 : Fixed NPE when using non-numeric range query in deleteByQuery
          Hide
          hossman Hoss Man added a comment -

          thanks for reporting this anders!

          Show
          hossman Hoss Man added a comment - thanks for reporting this anders!
          Hide
          mikemccand Michael McCandless added a comment -

          Bulk close resolved issues after 6.2.0 release.

          Show
          mikemccand Michael McCandless added a comment - Bulk close resolved issues after 6.2.0 release.

            People

            • Assignee:
              hossman Hoss Man
              Reporter:
              andersm Anders Melchiorsen
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development