Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 5.3
    • Fix Version/s: 5.3.2, 5.4.1, 5.5
    • Component/s: None
    • Labels:
      None

      Description

      The following URL select?debug=true&q=:&fl=id,score yields

      java.lang.NullPointerException
      	at org.apache.solr.handler.component.DebugComponent.finishStage(DebugComponent.java:229)
      	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:416)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:143)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2068)
      	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:669)
      	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:462)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:210)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:179)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
      	at org.eclipse.jetty.server.Server.handle(Server.java:499)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
      	at org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
      	at java.lang.Thread.run(Thread.java:745)
      
      

      I can reproduce it everytime. Strange enough fl=*,score, or any other content field does not! I have seen this happening in Highlighter as well on the same code path. It makes little sense, how would fl influence that piece of code, the id is requested in fl afterall.

      1. SOLR_8059.patch
        10 kB
        David Smiley

        Issue Links

          Activity

          Hide
          Jeff Wartes added a comment -

          I've seen this too. I assumed it was related to
          https://issues.apache.org/jira/browse/SOLR-1880, but I've never investigated.

          Show
          Jeff Wartes added a comment - I've seen this too. I assumed it was related to https://issues.apache.org/jira/browse/SOLR-1880 , but I've never investigated.
          Hide
          Shalin Shekhar Mangar added a comment -

          Yes, Jeff, I suspect the same looking at how only fl=id,score cause this.

          Show
          Shalin Shekhar Mangar added a comment - Yes, Jeff, I suspect the same looking at how only fl=id,score cause this.
          Hide
          Mani added a comment -

          In addition to the above this can be narrowed down to without looking at the source code fl=id,score & debug=results NPE is thrown.

          Steps to reproduce
          ./bin/solr start -e cloud #2 shards, 1 replica
          ./bin/solr status
          ./bin/post -c techproducts example/exampledocs/*.xml

          curl "http://localhost:8983/solr/techproducts/select?q=*%3A*&wt=json&indent=true&debug=true&fl=id,score" => NPE thrown
          curl "http://localhost:8983/solr/techproducts/select?q=*%3A*&wt=ruby&indent=true&debug=results&fl=id,score" => NPE thrown
          

          Following curl examples works fine
          curl "http://localhost:8983/solr/techproducts/select?q=%3A&wt=ruby&indent=true&debug=query&fl=id,score"
          curl "http://localhost:8983/solr/techproducts/select?q=%3A&wt=ruby&indent=true&debug=timing&fl=id,score"
          curl "http://localhost:8983/solr/techproducts/select?q=%3A&wt=ruby&indent=true&debug=track&fl=id,score"

          Show
          Mani added a comment - In addition to the above this can be narrowed down to without looking at the source code fl=id,score & debug=results NPE is thrown. Steps to reproduce ./bin/solr start -e cloud #2 shards, 1 replica ./bin/solr status ./bin/post -c techproducts example/exampledocs/*.xml curl "http://localhost:8983/solr/techproducts/select?q=*%3A*&wt=json&indent=true&debug=true&fl=id,score" => NPE thrown curl "http://localhost:8983/solr/techproducts/select?q=*%3A*&wt=ruby&indent=true&debug=results&fl=id,score" => NPE thrown Following curl examples works fine curl "http://localhost:8983/solr/techproducts/select?q= %3A &wt=ruby&indent=true&debug=query&fl=id,score" curl "http://localhost:8983/solr/techproducts/select?q= %3A &wt=ruby&indent=true&debug=timing&fl=id,score" curl "http://localhost:8983/solr/techproducts/select?q= %3A &wt=ruby&indent=true&debug=track&fl=id,score"
          Hide
          ASF GitHub Bot added a comment -

          GitHub user manisnesan opened a pull request:

          https://github.com/apache/lucene-solr/pull/213

          SOLR-8059 - NPE distributed DebugComponent. Test reproducer

          This patch reproduces the NPE exception thrown when fl=id query param used along with debug=true or debug=results.

          Example
          curl "http://localhost:8983/solr/techproducts/select?q=%3A&wt=json&indent=true&debug=true&fl=id,score" => NPE thrown
          curl "http://localhost:8983/solr/techproducts/select?q=%3A&wt=ruby&indent=true&debug=results&fl=id,score" => NPE thrown

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/manisnesan/lucene-solr trunk

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/lucene-solr/pull/213.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #213


          commit d28cd093a907446a632ede1a2838f0f405b1cfa9
          Author: Mani <msivanes@redhat.com>
          Date: 2015-12-02T03:27:15Z

          SOLR-8059 - NPE distributed DebugComponent. Test to reproduce NPE in DebugComponent.


          Show
          ASF GitHub Bot added a comment - GitHub user manisnesan opened a pull request: https://github.com/apache/lucene-solr/pull/213 SOLR-8059 - NPE distributed DebugComponent. Test reproducer This patch reproduces the NPE exception thrown when fl=id query param used along with debug=true or debug=results. Example curl "http://localhost:8983/solr/techproducts/select?q= %3A &wt=json&indent=true&debug=true&fl=id,score" => NPE thrown curl "http://localhost:8983/solr/techproducts/select?q= %3A &wt=ruby&indent=true&debug=results&fl=id,score" => NPE thrown You can merge this pull request into a Git repository by running: $ git pull https://github.com/manisnesan/lucene-solr trunk Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/213.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #213 commit d28cd093a907446a632ede1a2838f0f405b1cfa9 Author: Mani <msivanes@redhat.com> Date: 2015-12-02T03:27:15Z SOLR-8059 - NPE distributed DebugComponent. Test to reproduce NPE in DebugComponent.
          Hide
          Mani added a comment -

          Shalin Shekhar Mangar Since I do not have enough context/understanding about the DebugComponent, I have just updated with the reproducer. If you have any comments with the test let me know I can submit an updated patch.

          Show
          Mani added a comment - Shalin Shekhar Mangar Since I do not have enough context/understanding about the DebugComponent, I have just updated with the reproducer. If you have any comments with the test let me know I can submit an updated patch.
          Hide
          Christine Poerschke added a comment -

          Hi Mani - thanks for your steps to reproduce and pull request. SOLR-8059 here is already linked to SOLR-8060 and it appears that SOLR-8060 might already be fixed by SOLR-8340 ticket. Just mentioning in case the SOLR-8340 analysis and patch would help towards a patch for this SOLR-8059 here then.

          Show
          Christine Poerschke added a comment - Hi Mani - thanks for your steps to reproduce and pull request. SOLR-8059 here is already linked to SOLR-8060 and it appears that SOLR-8060 might already be fixed by SOLR-8340 ticket. Just mentioning in case the SOLR-8340 analysis and patch would help towards a patch for this SOLR-8059 here then.
          Hide
          David Smiley added a comment -

          I've worked on this a little. Locally I've enhanced the DistributedDebugComponentTest.testCompareWithNonDistributedRequest with some randomness that will trigger the single-pass query. In the process to get the distrib vs non-distrib equality tests to pass I've made some small enhancements to DebugComponent's merge logic so that it's more consistent between distributed & non-distributed, such as by merging the parsed filter query strings as a set instead of not as a set. Interestingly the one-pass optimization causes a difference in results. Shalin, please consider assigning this issue to me if you haven't invested time in it.

          Show
          David Smiley added a comment - I've worked on this a little. Locally I've enhanced the DistributedDebugComponentTest.testCompareWithNonDistributedRequest with some randomness that will trigger the single-pass query. In the process to get the distrib vs non-distrib equality tests to pass I've made some small enhancements to DebugComponent's merge logic so that it's more consistent between distributed & non-distributed, such as by merging the parsed filter query strings as a set instead of not as a set. Interestingly the one-pass optimization causes a difference in results. Shalin, please consider assigning this issue to me if you haven't invested time in it.
          Hide
          Shalin Shekhar Mangar added a comment -

          It's all yours David.

          Show
          Shalin Shekhar Mangar added a comment - It's all yours David.
          Hide
          David Smiley added a comment -

          The attached patch fixes the issue, and modifies an existing test that triggers this bug when the patch isn't applied. Furthermore, in this patch I extracted a utility method containing the affected buggy lines and put it into SolrPluginUtils under the method name copyNamedListIntoArrayByDocPosInResponse(), and modified HighlightComponent and TermVectorComponent to use this.

          I also tweaked DebugComponent's shard response merge logic to merge Collections (Lists typically) as if they were a Set. This fixes a minor response inconsistency between sharded and non-sharded, which I discovered when working on this test (and this test would fail if this were not fixed).

          I'll commit in a couple days, subject to feedback.

          Show
          David Smiley added a comment - The attached patch fixes the issue, and modifies an existing test that triggers this bug when the patch isn't applied. Furthermore, in this patch I extracted a utility method containing the affected buggy lines and put it into SolrPluginUtils under the method name copyNamedListIntoArrayByDocPosInResponse(), and modified HighlightComponent and TermVectorComponent to use this. I also tweaked DebugComponent's shard response merge logic to merge Collections (Lists typically) as if they were a Set. This fixes a minor response inconsistency between sharded and non-sharded, which I discovered when working on this test (and this test would fail if this were not fixed). I'll commit in a couple days, subject to feedback.
          Hide
          ASF subversion and git services added a comment -

          Commit 1721203 from David Smiley in branch 'dev/trunk'
          [ https://svn.apache.org/r1721203 ]

          SOLR-8059: debug=results&distrib.singlePass=true can NPE

          Show
          ASF subversion and git services added a comment - Commit 1721203 from David Smiley in branch 'dev/trunk' [ https://svn.apache.org/r1721203 ] SOLR-8059 : debug=results&distrib.singlePass=true can NPE
          Hide
          ASF subversion and git services added a comment -

          Commit 1721213 from David Smiley in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1721213 ]

          SOLR-8059: debug=results&distrib.singlePass=true can NPE

          Show
          ASF subversion and git services added a comment - Commit 1721213 from David Smiley in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1721213 ] SOLR-8059 : debug=results&distrib.singlePass=true can NPE
          Hide
          David Smiley added a comment -

          Closing; thanks Markus.

          It'd be nice to get this bug fix out into v5.3.2 perhaps although I'm not sure how in CHANGES.txt this entry would be recorded.

          Show
          David Smiley added a comment - Closing; thanks Markus. It'd be nice to get this bug fix out into v5.3.2 perhaps although I'm not sure how in CHANGES.txt this entry would be recorded.
          Hide
          ASF subversion and git services added a comment -

          Commit 1721931 from David Smiley in branch 'dev/branches/lucene_solr_5_3'
          [ https://svn.apache.org/r1721931 ]

          SOLR-8059 SOLR-8340: Fix HighlightComponent & DebugComponent finishStage NPE's when distrib.singlePass

          Show
          ASF subversion and git services added a comment - Commit 1721931 from David Smiley in branch 'dev/branches/lucene_solr_5_3' [ https://svn.apache.org/r1721931 ] SOLR-8059 SOLR-8340 : Fix HighlightComponent & DebugComponent finishStage NPE's when distrib.singlePass
          Hide
          ASF subversion and git services added a comment -

          Commit 1722064 from Anshum Gupta in branch 'dev/trunk'
          [ https://svn.apache.org/r1722064 ]

          SOLR-8059: Add change log entry to 5.3.2 section on trunk

          Show
          ASF subversion and git services added a comment - Commit 1722064 from Anshum Gupta in branch 'dev/trunk' [ https://svn.apache.org/r1722064 ] SOLR-8059 : Add change log entry to 5.3.2 section on trunk
          Hide
          ASF subversion and git services added a comment -

          Commit 1722065 from Anshum Gupta in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1722065 ]

          SOLR-8059: Add change log entry to 5.3.2 section (merge from trunk)

          Show
          ASF subversion and git services added a comment - Commit 1722065 from Anshum Gupta in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1722065 ] SOLR-8059 : Add change log entry to 5.3.2 section (merge from trunk)
          Hide
          ASF subversion and git services added a comment -

          Commit 1724180 from Adrien Grand in branch 'dev/branches/lucene_solr_5_4'
          [ https://svn.apache.org/r1724180 ]

          SOLR-8059: debug=results&distrib.singlePass=true can NPE

          Show
          ASF subversion and git services added a comment - Commit 1724180 from Adrien Grand in branch 'dev/branches/lucene_solr_5_4' [ https://svn.apache.org/r1724180 ] SOLR-8059 : debug=results&distrib.singlePass=true can NPE
          Hide
          David Smiley added a comment -

          Adrien Grand the CHANGES.txt you added is for SOLR-8050, not SOLR-8059. I'm getting pretty confused. Furthermore, shouldn't the CHANGES.txt entry be in 5.4.1 in addition to 5.3.2? After all, it's not in 5.4.0 so nobody casual user would realize 5.4.1 has these fixes. Further confusing matters is that the CHANGES.txt is a little out of sync between branch lucene_solr_5_4 and lucene_solr_5_3 – the latter shows this change (SOLR-8059) in the right spot whereas in lucene_solr_5_4 it shows SOLR-8050 in its place.

          Show
          David Smiley added a comment - Adrien Grand the CHANGES.txt you added is for SOLR-8050 , not SOLR-8059 . I'm getting pretty confused. Furthermore, shouldn't the CHANGES.txt entry be in 5.4.1 in addition to 5.3.2? After all, it's not in 5.4.0 so nobody casual user would realize 5.4.1 has these fixes. Further confusing matters is that the CHANGES.txt is a little out of sync between branch lucene_solr_5_4 and lucene_solr_5_3 – the latter shows this change ( SOLR-8059 ) in the right spot whereas in lucene_solr_5_4 it shows SOLR-8050 in its place.
          Hide
          Adrien Grand added a comment - - edited

          Thanks David, something went wrong indeed. I supect that I kept the wrong CHANGES entry while backporting (because of conflicts in solr/CHANGES) and when I later looked into which changes need to be listed under 5.4.1 I saw that SOLR-8059 was already in 5.4.0 and thus did not deserve an entry under 5.4.1. I'll review the changes that I did and fix.

          Show
          Adrien Grand added a comment - - edited Thanks David, something went wrong indeed. I supect that I kept the wrong CHANGES entry while backporting (because of conflicts in solr/CHANGES) and when I later looked into which changes need to be listed under 5.4.1 I saw that SOLR-8059 was already in 5.4.0 and thus did not deserve an entry under 5.4.1. I'll review the changes that I did and fix.
          Hide
          ASF subversion and git services added a comment -

          Commit 1724263 from Adrien Grand in branch 'dev/branches/lucene_solr_5_4'
          [ https://svn.apache.org/r1724263 ]

          SOLR-8058, SOLR-8059: Fix CHANGES entries.

          Show
          ASF subversion and git services added a comment - Commit 1724263 from Adrien Grand in branch 'dev/branches/lucene_solr_5_4' [ https://svn.apache.org/r1724263 ] SOLR-8058 , SOLR-8059 : Fix CHANGES entries.
          Hide
          Adrien Grand added a comment -

          I reviewed the changes and as far as I can tell the problem was only with the solr/CHANGES.txt file. If you could confirm, David, I would appreciate!

          Show
          Adrien Grand added a comment - I reviewed the changes and as far as I can tell the problem was only with the solr/CHANGES.txt file. If you could confirm, David, I would appreciate!
          Hide
          David Smiley added a comment -

          Looks good Adrien; thanks!

          Show
          David Smiley added a comment - Looks good Adrien; thanks!
          Hide
          ASF subversion and git services added a comment -

          Commit d83223a564f0c4e2b09af22f8dcf139cc415ce5c in lucene-solr's branch refs/heads/branch_5_4 from Adrien Grand
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d83223a ]

          SOLR-8058, SOLR-8059: Fix CHANGES entries.

          git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_4@1724263 13f79535-47bb-0310-9956-ffa450edef68

          Show
          ASF subversion and git services added a comment - Commit d83223a564f0c4e2b09af22f8dcf139cc415ce5c in lucene-solr's branch refs/heads/branch_5_4 from Adrien Grand [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d83223a ] SOLR-8058 , SOLR-8059 : Fix CHANGES entries. git-svn-id: https://svn.apache.org/repos/asf/lucene/dev/branches/lucene_solr_5_4@1724263 13f79535-47bb-0310-9956-ffa450edef68

            People

            • Assignee:
              David Smiley
              Reporter:
              Markus Jelsma
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development