Solr
  1. Solr
  2. SOLR-2121

distributed highlighting using q.alt=*:* causes NPE in finishStages

    Details

    • Type: Bug Bug
    • Status: Reopened
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      As noted on the mailing list by Ron Mayer, using the example configs and example data on trunk, this query works...

      http://localhost:8983/solr/select?q.alt=*:*&hl=on&defType=edismax

      ...but this query causes and NullPointerException...

      http://localhost:8983/solr/select?q.alt=*:*&hl=on&defType=edismax&shards=localhost:8983/solr

      Stack Trace...

      java.lang.NullPointerException
      	at org.apache.solr.handler.component.HighlightComponent.finishStage(HighlightComponent.java:158)
      	at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:310)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:1324)
      	at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:337)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:240)
      	at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
      
      

        Activity

        Hide
        Hoss Man added a comment -

        Marc Sturlese posted his "fix" but it's not entirely obvious to me what exactly the necessary change is, or if the root cause isn't somewhere else...

          public void finishStage(ResponseBuilder rb) {
            boolean hasHighlighting = true ;
            if (rb.doHighlights && rb.stage == ResponseBuilder.STAGE_GET_FIELDS) {
        
              Map.Entry<String, Object>[] arr = new
        NamedList.NamedListEntry[rb.resultIds.size()];
        
              // TODO: make a generic routine to do automatic merging of id keyed
        data
              for (ShardRequest sreq : rb.finished) {
                if ((sreq.purpose & ShardRequest.PURPOSE_GET_HIGHLIGHTS) == 0)
        continue;
                for (ShardResponse srsp : sreq.responses) {
                  NamedList hl =
        (NamedList)srsp.getSolrResponse().getResponse().get("highlighting");
                  //patch bug
                  if(hl != null) {
                    for (int i=0; i<hl.size(); i++) {
                     String id = hl.getName(i);
                     ShardDoc sdoc = rb.resultIds.get(id);
                     int idx = sdoc.positionInResponse;
                     arr[idx] = new NamedList.NamedListEntry(id, hl.getVal(i));
                    }
                  } else {
                    hasHighlighting = false;
                  }
                }
              }
        
              // remove nulls in case not all docs were able to be retrieved
              //patch bug
              if(hasHighlighting) {
                rb.rsp.add("highlighting", removeNulls(new SimpleOrderedMap(arr)));
              }
            }
          }
        
        
        Show
        Hoss Man added a comment - Marc Sturlese posted his "fix" but it's not entirely obvious to me what exactly the necessary change is, or if the root cause isn't somewhere else... public void finishStage(ResponseBuilder rb) { boolean hasHighlighting = true ; if (rb.doHighlights && rb.stage == ResponseBuilder.STAGE_GET_FIELDS) { Map.Entry< String , Object >[] arr = new NamedList.NamedListEntry[rb.resultIds.size()]; // TODO: make a generic routine to do automatic merging of id keyed data for (ShardRequest sreq : rb.finished) { if ((sreq.purpose & ShardRequest.PURPOSE_GET_HIGHLIGHTS) == 0) continue ; for (ShardResponse srsp : sreq.responses) { NamedList hl = (NamedList)srsp.getSolrResponse().getResponse().get( "highlighting" ); //patch bug if (hl != null ) { for ( int i=0; i<hl.size(); i++) { String id = hl.getName(i); ShardDoc sdoc = rb.resultIds.get(id); int idx = sdoc.positionInResponse; arr[idx] = new NamedList.NamedListEntry(id, hl.getVal(i)); } } else { hasHighlighting = false ; } } } // remove nulls in case not all docs were able to be retrieved //patch bug if (hasHighlighting) { rb.rsp.add( "highlighting" , removeNulls( new SimpleOrderedMap(arr))); } } }
        Hide
        Andrey Strizhkin added a comment -

        This fix only avoid NPE but highlighting doesn't work.

        Improved fix that works for me:

        public void finishStage(ResponseBuilder rb) {
            boolean hasHighlighting = false;
            if (rb.doHighlights && rb.stage == ResponseBuilder.STAGE_GET_FIELDS) {
        
              Map.Entry<String, Object>[] arr = new
        NamedList.NamedListEntry[rb.resultIds.size()];
        
              // TODO: make a generic routine to do automatic merging of id keyed
        data
              for (ShardRequest sreq : rb.finished) {
                if ((sreq.purpose & ShardRequest.PURPOSE_GET_HIGHLIGHTS) == 0)
        continue;
                for (ShardResponse srsp : sreq.responses) {
                  NamedList hl =
        (NamedList)srsp.getSolrResponse().getResponse().get("highlighting");
                  //patch bug
                  if(hl != null) {
                    hasHighlighting = true;
                    for (int i=0; i<hl.size(); i++) {
                     String id = hl.getName(i);
                     ShardDoc sdoc = rb.resultIds.get(id);
                     int idx = sdoc.positionInResponse;
                     arr[idx] = new NamedList.NamedListEntry(id, hl.getVal(i));
                    }
                  } 
                }
              }
        
              // remove nulls in case not all docs were able to be retrieved
              //patch bug
              if(hasHighlighting) {
                rb.rsp.add("highlighting", removeNulls(new SimpleOrderedMap(arr)));
              }
            }
          }
        
        Show
        Andrey Strizhkin added a comment - This fix only avoid NPE but highlighting doesn't work. Improved fix that works for me: public void finishStage(ResponseBuilder rb) { boolean hasHighlighting = false ; if (rb.doHighlights && rb.stage == ResponseBuilder.STAGE_GET_FIELDS) { Map.Entry< String , Object >[] arr = new NamedList.NamedListEntry[rb.resultIds.size()]; // TODO: make a generic routine to do automatic merging of id keyed data for (ShardRequest sreq : rb.finished) { if ((sreq.purpose & ShardRequest.PURPOSE_GET_HIGHLIGHTS) == 0) continue ; for (ShardResponse srsp : sreq.responses) { NamedList hl = (NamedList)srsp.getSolrResponse().getResponse().get( "highlighting" ); //patch bug if (hl != null ) { hasHighlighting = true ; for ( int i=0; i<hl.size(); i++) { String id = hl.getName(i); ShardDoc sdoc = rb.resultIds.get(id); int idx = sdoc.positionInResponse; arr[idx] = new NamedList.NamedListEntry(id, hl.getVal(i)); } } } } // remove nulls in case not all docs were able to be retrieved //patch bug if (hasHighlighting) { rb.rsp.add( "highlighting" , removeNulls( new SimpleOrderedMap(arr))); } } }
        Hide
        Koji Sekiguchi added a comment -

        ...but this query causes and NullPointerException...

        http://localhost:8983/solr/select?q.alt=*:*&hl=on&defType=edismax&shards=localhost:8983/solr

        I cannot reproduce the problem with the latest trunk. I think SOLR-2148 solved this problem.

        Show
        Koji Sekiguchi added a comment - ...but this query causes and NullPointerException... http://localhost:8983/solr/select?q.alt=*:*&hl=on&defType=edismax&shards=localhost:8983/solr I cannot reproduce the problem with the latest trunk. I think SOLR-2148 solved this problem.
        Hide
        Koji Sekiguchi added a comment -

        Close as duplicate of SOLR-1901.

        Show
        Koji Sekiguchi added a comment - Close as duplicate of SOLR-1901 .
        Hide
        Dylan Etkin added a comment -

        This issue still exists in solr 3.1.0.

        The code snippet in the second comment does stop the NPE from happening.

        Show
        Dylan Etkin added a comment - This issue still exists in solr 3.1.0. The code snippet in the second comment does stop the NPE from happening.
        Hide
        Hoss Man added a comment -

        Dylan: koji wasn't able to reproduce the initial report after those related fixes.

        if you are still seeing this problem, can you post a URL for the example configs that triggers the exception for you? (or attach a test case of some other configs that you cna use to trigger this)

        Show
        Hoss Man added a comment - Dylan: koji wasn't able to reproduce the initial report after those related fixes. if you are still seeing this problem, can you post a URL for the example configs that triggers the exception for you? (or attach a test case of some other configs that you cna use to trigger this)

          People

          • Assignee:
            Koji Sekiguchi
            Reporter:
            Hoss Man
          • Votes:
            1 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:

              Development