Solr
  1. Solr
  2. SOLR-3256

Distributed search throws NPE when using fl=score

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0-ALPHA
    • Component/s: None
    • Labels:
      None

      Description

      Steps to reproduce the problem:
      Start two Solr instances (may use the example configuration)
      add some documents to both instances
      execute a query like: http://localhost:8983/solr/select?shards=localhost:8983/solr,localhost:8984/solr&q=(ipod%20OR%20display)&*fl=score*

      Expected result:
      List of scores or at least an exception saying that this request is not supported (may not make too much sense to do fl=score, but a descriptive exception can help debug the problem)

      Getting:
      SEVERE: null:java.lang.NullPointerException
      at org.apache.solr.handler.component.QueryComponent.returnFields(QueryComponent.java:985)
      at org.apache.solr.handler.component.QueryComponent.handleRegularResponses(QueryComponent.java:637)
      at org.apache.solr.handler.component.QueryComponent.handleResponses(QueryComponent.java:612)
      at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:307)
      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
      at org.apache.solr.core.SolrCore.execute(SolrCore.java:1540)
      at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:435)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:256)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1337)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:484)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:119)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:524)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:233)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1065)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:413)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:192)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:999)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:250)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:149)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:111)
      at org.eclipse.jetty.server.Server.handle(Server.java:351)
      at org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:454)
      at org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:47)
      at org.eclipse.jetty.server.AbstractHttpConnection.headerComplete(AbstractHttpConnection.java:890)
      at org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.headerComplete(AbstractHttpConnection.java:944)
      at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:634)
      at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:230)
      at org.eclipse.jetty.server.BlockingHttpConnection.handle(BlockingHttpConnection.java:66)
      at org.eclipse.jetty.server.bio.SocketConnector$ConnectorEndPoint.run(SocketConnector.java:254)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:599)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:534)
      at java.lang.Thread.run(Thread.java:636)

      1. SOLR-3256.patch
        3 kB
        Tomás Fernández Löbbe
      2. SOLR-3256.patch
        0.8 kB
        Tomás Fernández Löbbe

        Issue Links

          Activity

          Hide
          Yonik Seeley added a comment -
          Show
          Yonik Seeley added a comment - Confirmed... and you don't even need two nodes: http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=score
          Hide
          Yonik Seeley added a comment -

          It looks like the fl is modified for the first phase to "id,score" but not on the second phase.

          INFO: [] webapp=/solr path=/select params={fl=id,score&shard.url=localhost:8983/solr&NOW=1332206350114&start=0&q=*:*&distrib=false&isShard=true&wt=javabin&fsv=true&rows=10&version=2} hits=32 status=0 QTime=1 
          Mar 19, 2012 9:19:10 PM org.apache.solr.core.SolrCore execute
          INFO: [] webapp=/solr path=/select params={fl=score&shard.url=localhost:8983/solr&NOW=1332206350114&q=*:*&ids=SP2514N,GB18030TEST,apple,F8V7067-APL-KIT,adata,6H500F0,MA147LL/A,ati,IW-02,asus&distrib=false&isShard=true&wt=javabin&rows=10&version=2} status=0 QTime=1 
          
          Show
          Yonik Seeley added a comment - It looks like the fl is modified for the first phase to "id,score" but not on the second phase. INFO: [] webapp=/solr path=/select params={fl=id,score&shard.url=localhost:8983/solr&NOW=1332206350114&start=0&q=*:*&distrib= false &isShard= true &wt=javabin&fsv= true &rows=10&version=2} hits=32 status=0 QTime=1 Mar 19, 2012 9:19:10 PM org.apache.solr.core.SolrCore execute INFO: [] webapp=/solr path=/select params={fl=score&shard.url=localhost:8983/solr&NOW=1332206350114&q=*:*&ids=SP2514N,GB18030TEST,apple,F8V7067-APL-KIT,adata,6H500F0,MA147LL/A,ati,IW-02,asus&distrib= false &isShard= true &wt=javabin&rows=10&version=2} status=0 QTime=1
          Hide
          Yonik Seeley added a comment -

          Culprit is line 948 of QueryComponent:

                  // currently, "score" is synonymous with "*,score" so
                  // don't add "id" if the fl is empty or "score" or it would change the meaning.
                   if (fl.length()!=0 && !"score".equals(fl) && !"*".equals(fl)) {
                     sreq.params.set(CommonParams.FL, fl+','+uniqueField.getName());
                   }
          

          I think with the new pseudo-fields / ReturnFields feature, "score" no longer means "*,score".

          Furthermore, it doesn't look like the "multiple fl param" feature is working with distributed search:

          http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=id&fl=price&fl=cat
          (the response only contains "id")

          Show
          Yonik Seeley added a comment - Culprit is line 948 of QueryComponent: // currently, "score" is synonymous with "*,score" so // don't add "id" if the fl is empty or "score" or it would change the meaning. if (fl.length()!=0 && ! "score" .equals(fl) && ! "*" .equals(fl)) { sreq.params.set(CommonParams.FL, fl+','+uniqueField.getName()); } I think with the new pseudo-fields / ReturnFields feature, "score" no longer means "*,score". Furthermore, it doesn't look like the "multiple fl param" feature is working with distributed search: http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=id&fl=price&fl=cat (the response only contains "id")
          Hide
          Tomás Fernández Löbbe added a comment -

          It's rare, it seems to depend on the order of the fl parameters.
          http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=id&fl=cat&fl=price

          shows only the id,

          http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=cat&fl=id&fl=price

          shows id and cat and
          http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=price&fl=cat&fl=id
          shows price and id.

          I'm attaching a patch that demonstrates the failure with a test case.

          Show
          Tomás Fernández Löbbe added a comment - It's rare, it seems to depend on the order of the fl parameters. http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=id&fl=cat&fl=price shows only the id, http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=cat&fl=id&fl=price shows id and cat and http://localhost:8983/solr/select?q=*:*&shards=localhost:8983/solr&fl=price&fl=cat&fl=id shows price and id. I'm attaching a patch that demonstrates the failure with a test case.
          Hide
          Luca Cavanna added a comment -

          Regarding the legacy behavior fl=score which was equals to fl=*,score : it has been removed from trunk a few weeks ago (SOLR-2712).

          Show
          Luca Cavanna added a comment - Regarding the legacy behavior fl=score which was equals to fl=*,score : it has been removed from trunk a few weeks ago ( SOLR-2712 ).
          Hide
          Tomás Fernández Löbbe added a comment -

          I'm attaching a possible fix. It doesn't assumes that "score" means "*,score" anymore. It also fixes the problem with the order of the "fl" parameters when they are more than one.

          Show
          Tomás Fernández Löbbe added a comment - I'm attaching a possible fix. It doesn't assumes that "score" means "*,score" anymore. It also fixes the problem with the order of the "fl" parameters when they are more than one.
          Hide
          Mark Miller added a comment -

          Hey Tomas - this has an overlap with SOLR-3214 - and it looks like this fix covers the SOLR-3214 fix - can you confirm so we can close them both when this goes in?

          Show
          Mark Miller added a comment - Hey Tomas - this has an overlap with SOLR-3214 - and it looks like this fix covers the SOLR-3214 fix - can you confirm so we can close them both when this goes in?
          Hide
          Tomás Fernández Löbbe added a comment -

          You are right Mark, it does overlap with SOLR-3214. This fix seems to fix that too. I added this lines to the TestDistributedSearch:

          query("q",":", "fl","score","sort",i1 + " desc");
          query("q",":", "fl",i1 + ",score","sort",i1 + " desc");
          query("q",":", "fl", i1, "fl","score","sort",i1 + " desc");
          query("q",":", "fl", "id," + i1,"sort",i1 + " desc");
          query("q",":", "fl", "id", "fl",i1,"sort",i1 + " desc");
          query("q",":", "fl",i1, "fl", "id","sort",i1 + " desc");
          query("q",":", "fl", "id", "fl",nint, "fl",tint,"sort",i1 + " desc");
          query("q",":", "fl",nint, "fl", "id", "fl",tint,"sort",i1 + " desc");

          Those work with my patch. I also manually tested the requests I commented before and those work OK too.

          Show
          Tomás Fernández Löbbe added a comment - You are right Mark, it does overlap with SOLR-3214 . This fix seems to fix that too. I added this lines to the TestDistributedSearch: query("q"," : ", "fl","score","sort",i1 + " desc"); query("q"," : ", "fl",i1 + ",score","sort",i1 + " desc"); query("q"," : ", "fl", i1, "fl","score","sort",i1 + " desc"); query("q"," : ", "fl", "id," + i1,"sort",i1 + " desc"); query("q"," : ", "fl", "id", "fl",i1,"sort",i1 + " desc"); query("q"," : ", "fl",i1, "fl", "id","sort",i1 + " desc"); query("q"," : ", "fl", "id", "fl",nint, "fl",tint,"sort",i1 + " desc"); query("q"," : ", "fl",nint, "fl", "id", "fl",tint,"sort",i1 + " desc"); Those work with my patch. I also manually tested the requests I commented before and those work OK too.
          Hide
          Mark Miller added a comment -

          Thanks Tomas!

          Show
          Mark Miller added a comment - Thanks Tomas!

            People

            • Assignee:
              Mark Miller
              Reporter:
              Tomás Fernández Löbbe
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development