Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.3
    • Fix Version/s: 6.4, 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      I've spotted this before, and just now one of my unit tests does it as well. I believe this happens when there is no data for the requested field.

      java.lang.NullPointerException
              at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
              at org.apache.solr.search.facet.PercentileAgg$Merger.merge(PercentileAgg.java:195)
              at org.apache.solr.search.facet.FacetBucket.mergeBucket(FacetBucket.java:90)
              at org.apache.solr.search.facet.FacetRequestSortedMerger.mergeBucketList(FacetRequestSortedMerger.java:61)
              at org.apache.solr.search.facet.FacetRangeMerger.mergeBucketList(FacetRangeMerger.java:27)
              at org.apache.solr.search.facet.FacetRangeMerger.merge(FacetRangeMerger.java:91)
              at org.apache.solr.search.facet.FacetRangeMerger.merge(FacetRangeMerger.java:43)
              at org.apache.solr.search.facet.FacetBucket.mergeBucket(FacetBucket.java:90)
              at org.apache.solr.search.facet.FacetQueryMerger.merge(FacetModule.java:444)
              at org.apache.solr.search.facet.FacetModule.handleResponses(FacetModule.java:272)
      
      1. SOLR-9917.patch
        9 kB
        Yonik Seeley
      2. SOLR-9917.patch
        4 kB
        Yonik Seeley

        Activity

        Hide
        markus17 Markus Jelsma added a comment -

        I can now also confirm that lack of data for a field causes the NPE to occur. I added data for a field to my failing unit test, it now passes.

        Show
        markus17 Markus Jelsma added a comment - I can now also confirm that lack of data for a field causes the NPE to occur. I added data for a field to my failing unit test, it now passes.
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Thanks for investigating Markus, I'll look into it now...

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Thanks for investigating Markus, I'll look into it now...
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        It's odd this never happens once in a while to TestJsonFacets, so the criteria may be more specific than having a shard w/o a field. What does the relevant part of the query request look like (i.e. how is the percentile being used and what is it nested under?)

        Show
        yseeley@gmail.com Yonik Seeley added a comment - It's odd this never happens once in a while to TestJsonFacets, so the criteria may be more specific than having a shard w/o a field. What does the relevant part of the query request look like (i.e. how is the percentile being used and what is it nested under?)
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Here's a test patch I tried using to force empty shards. But I guess we need more along the lines of shards with documents but w/o the values. I'll try that next.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Here's a test patch I tried using to force empty shards. But I guess we need more along the lines of shards with documents but w/o the values. I'll try that next.
        Hide
        markus17 Markus Jelsma added a comment -

        I only have access to the code right now, here's the SolrJ code causing it:

        vars:
        logHistory is just the number of days we look back in the history;
        percentile is, well, the percentile, it is set to 90th percentile

            SolrQuery query = new SolrQuery("{!term f=document}" + document);
            query.setRows(0);
            query.addFilterQuery("{!term f=type}view");
            query.addFilterQuery("time:[NOW-" + logHistory + "DAY/DAY TO NOW+1DAY/DAY]");
        
            query.setParam("json.facet", "{period:{type:range,field:time,start:\"NOW-" + logHistory + "DAY/DAY\",end:\"NOW+1DAY/DAY\",gap:\"+" + logHistory + "DAY\",facet:{time_percentiles:\"percentile(view_time," + percentile + ")\"}}}");
        
            QueryResponse response = client.query(query);
        

        Let me know if you need more information.

        Show
        markus17 Markus Jelsma added a comment - I only have access to the code right now, here's the SolrJ code causing it: vars: logHistory is just the number of days we look back in the history; percentile is, well, the percentile, it is set to 90th percentile SolrQuery query = new SolrQuery( "{!term f=document}" + document); query.setRows(0); query.addFilterQuery( "{!term f=type}view" ); query.addFilterQuery( "time:[NOW-" + logHistory + "DAY/DAY TO NOW+1DAY/DAY]" ); query.setParam( "json.facet" , "{period:{type:range,field:time,start:\" NOW- " + logHistory + " DAY/DAY\ ",end:\" NOW+1DAY/DAY\ ",gap:\" + " + logHistory + " DAY\ ",facet:{time_percentiles:\" percentile(view_time, " + percentile + " )\ "}}}" ); QueryResponse response = client.query(query); Let me know if you need more information.
        Hide
        markus17 Markus Jelsma added a comment -

        Oh, i should add that this is indeed in cloud mode. The unit tests also run in cloud mode.

        Show
        markus17 Markus Jelsma added a comment - Oh, i should add that this is indeed in cloud mode. The unit tests also run in cloud mode.
        Hide
        markus17 Markus Jelsma added a comment - - edited

        The NPE is thrown at the line where i execute the query with this test input data:

            indexDoc(sdoc("document", "document_A", "uid", "1", "time", time(24), "type", "view", "key", "test_key", "coord", "53.22,6.56"));
        

        It's gone when adding the requested data (view_time=50)

            indexDoc(sdoc("document", "document_A", "uid", "1", "time", time(24), "type", "view", "key", "test_key", "coord", "53.22,6.56", "view_time", "50"));
        

        Test extends AbstractFullDistribZkTestBase and has @ShardsFixed(num = 2) set

        Show
        markus17 Markus Jelsma added a comment - - edited The NPE is thrown at the line where i execute the query with this test input data: indexDoc(sdoc( "document" , "document_A" , "uid" , "1" , "time" , time(24), "type" , "view" , "key" , "test_key" , "coord" , "53.22,6.56" )); It's gone when adding the requested data (view_time=50) indexDoc(sdoc( "document" , "document_A" , "uid" , "1" , "time" , time(24), "type" , "view" , "key" , "test_key" , "coord" , "53.22,6.56" , "view_time" , "50" )); Test extends AbstractFullDistribZkTestBase and has @ShardsFixed(num = 2) set
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Thanks for the extra info, I've now reproduced it and am working on a fix.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Thanks for the extra info, I've now reproduced it and am working on a fix.
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        OK, here's a patch + test.

        Show
        yseeley@gmail.com Yonik Seeley added a comment - OK, here's a patch + test.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 7ef6a8184682e046a6a8ce9166978c320d285a1a in lucene-solr's branch refs/heads/master from Yonik Seeley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7ef6a81 ]

        SOLR-9917: fix NPE in distrib percentiles when no values for field in bucket

        Show
        jira-bot ASF subversion and git services added a comment - Commit 7ef6a8184682e046a6a8ce9166978c320d285a1a in lucene-solr's branch refs/heads/master from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7ef6a81 ] SOLR-9917 : fix NPE in distrib percentiles when no values for field in bucket
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 302cee59242c95b03750ebeb581b5ac27e8f2c55 in lucene-solr's branch refs/heads/branch_6x from Yonik Seeley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=302cee5 ]

        SOLR-9917: fix NPE in distrib percentiles when no values for field in bucket

        Show
        jira-bot ASF subversion and git services added a comment - Commit 302cee59242c95b03750ebeb581b5ac27e8f2c55 in lucene-solr's branch refs/heads/branch_6x from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=302cee5 ] SOLR-9917 : fix NPE in distrib percentiles when no values for field in bucket

          People

          • Assignee:
            yseeley@gmail.com Yonik Seeley
            Reporter:
            markus17 Markus Jelsma
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development