Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10383

NPE on debug query in SOLR UI - LTR OriginalScoreFeature

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.4, 6.4.1, 6.4.2, 6.5
    • Fix Version/s: 6.5.1, 6.6, 7.0
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      Hi,

      there is a NPE if I want to debug query in SOLR UI.

      I'm using LTR for reranking result.

      My features:

      {
        "initArgs":{},
        "initializedOn":"2017-03-29T05:32:52.160Z",
        "updatedSinceInit":"2017-03-29T05:56:28.721Z",
        "managedList":[
          {
            "name":"documentRecency",
            "class":"org.apache.solr.ltr.feature.SolrFeature",
            "params":{"q":"{!func}recip( ms(NOW,initial_release_date), 3.16e-11, 1, 1)"},
            "store":"_DEFAULT_"},
          {
            "name":"niceness",
            "class":"org.apache.solr.ltr.feature.SolrFeature",
            "params":{"fq":["{!func}recip(niceness, 0.1, 1, 1)"]},
            "store":"_DEFAULT_"},
          {
            "name":"originalScore",
            "class":"org.apache.solr.ltr.feature.OriginalScoreFeature",
            "params":null,
            "store":"_DEFAULT_"}]}
      

      My model:

      {
        "initArgs":{},
        "initializedOn":"2017-03-29T05:32:52.167Z",
        "updatedSinceInit":"2017-03-29T05:54:26.100Z",
        "managedList":[{
            "name":"myModel",
            "class":"org.apache.solr.ltr.model.LinearModel",
            "store":"_DEFAULT_",
            "features":[
              {
                "name":"documentRecency",
                "norm":{"class":"org.apache.solr.ltr.norm.IdentityNormalizer"}},
              {
                "name":"niceness",
                "norm":{"class":"org.apache.solr.ltr.norm.IdentityNormalizer"}},
              {
                "name":"originalScore",
                "norm":{"class":"org.apache.solr.ltr.norm.IdentityNormalizer"}}],
            "params":{"weights":{
                "documentRecency":0.1,
                "niceness":1.0,
                "originalScore":0.5}}}]}
      

      NPE occurs in this method, where docInfo is null.

      OriginalScoreFeature.java
      @Override
            public float score() throws IOException {
              // This is done to improve the speed of feature extraction. Since this
              // was already scored in step 1
              // we shouldn't need to calc original score again.
              final DocInfo docInfo = getDocInfo();
              return (docInfo.hasOriginalDocScore() ? docInfo.getOriginalDocScore() : originalScorer.score());
            }
      
      1. SOLR-10383.patch
        3 kB
        Christine Poerschke
      2. SOLR-10383-prep.patch
        8 kB
        Christine Poerschke
      3. SOLR-10383.patch
        2 kB
        Christine Poerschke

        Issue Links

          Activity

          Hide
          zaky.vit Vitezslav Zak added a comment -

          Nice work,

          thank you too.

          Best

          Show
          zaky.vit Vitezslav Zak added a comment - Nice work, thank you too. Best
          Hide
          cpoerschke Christine Poerschke added a comment -

          Thanks Vitezslav!

          Show
          cpoerschke Christine Poerschke added a comment - Thanks Vitezslav!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 94ca740f3b5c425c6b3f8d20864ab168794744ca in lucene-solr's branch refs/heads/branch_6_5 from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=94ca740 ]

          SOLR-10383: remove duplicate 6.5.1 section heading

          Show
          jira-bot ASF subversion and git services added a comment - Commit 94ca740f3b5c425c6b3f8d20864ab168794744ca in lucene-solr's branch refs/heads/branch_6_5 from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=94ca740 ] SOLR-10383 : remove duplicate 6.5.1 section heading
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit c9b3b265994480fe76c8d74dbdffa17054b3d91f in lucene-solr's branch refs/heads/branch_6_5 from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c9b3b26 ]

          SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class.
          (Vitezslav Zak, Christine Poerschke)

          Show
          jira-bot ASF subversion and git services added a comment - Commit c9b3b265994480fe76c8d74dbdffa17054b3d91f in lucene-solr's branch refs/heads/branch_6_5 from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=c9b3b26 ] SOLR-10383 : Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class. (Vitezslav Zak, Christine Poerschke)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit cf7f0c0670b2b670afb28fc93a72901d22473f7a in lucene-solr's branch refs/heads/branch_6_5 from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cf7f0c0 ]

          SOLR-10383: reduce code duplication in TestOriginalScoreFeature

          Show
          jira-bot ASF subversion and git services added a comment - Commit cf7f0c0670b2b670afb28fc93a72901d22473f7a in lucene-solr's branch refs/heads/branch_6_5 from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=cf7f0c0 ] SOLR-10383 : reduce code duplication in TestOriginalScoreFeature
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit ff840ad59c4eccc834e43c54194643a99a0f697a in lucene-solr's branch refs/heads/branch_6x from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ff840ad ]

          SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class.
          (Vitezslav Zak, Christine Poerschke)

          Show
          jira-bot ASF subversion and git services added a comment - Commit ff840ad59c4eccc834e43c54194643a99a0f697a in lucene-solr's branch refs/heads/branch_6x from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ff840ad ] SOLR-10383 : Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class. (Vitezslav Zak, Christine Poerschke)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 05cfe1345fd9e0e7514fbcf0df55ad248be981b0 in lucene-solr's branch refs/heads/branch_6x from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=05cfe13 ]

          SOLR-10383: reduce code duplication in TestOriginalScoreFeature

          Show
          jira-bot ASF subversion and git services added a comment - Commit 05cfe1345fd9e0e7514fbcf0df55ad248be981b0 in lucene-solr's branch refs/heads/branch_6x from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=05cfe13 ] SOLR-10383 : reduce code duplication in TestOriginalScoreFeature
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 186c5edd63fe292388ad435bc1cbb1a32a3a3824 in lucene-solr's branch refs/heads/master from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=186c5edd ]

          SOLR-10383: Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class.
          (Vitezslav Zak, Christine Poerschke)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 186c5edd63fe292388ad435bc1cbb1a32a3a3824 in lucene-solr's branch refs/heads/master from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=186c5edd ] SOLR-10383 : Fix debug related NullPointerException in solr/contrib/ltr OriginalScoreFeature class. (Vitezslav Zak, Christine Poerschke)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit e875f135bee21484386160b258b0eb6f0d2b7738 in lucene-solr's branch refs/heads/master from Christine Poerschke
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e875f13 ]

          SOLR-10383: reduce code duplication in TestOriginalScoreFeature

          Show
          jira-bot ASF subversion and git services added a comment - Commit e875f135bee21484386160b258b0eb6f0d2b7738 in lucene-solr's branch refs/heads/master from Christine Poerschke [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=e875f13 ] SOLR-10383 : reduce code duplication in TestOriginalScoreFeature
          Hide
          cpoerschke Christine Poerschke added a comment -

          Patch with naive fix and test change. (There might be another approach i.e. avoiding docInfo being non-null at that place in that code path but I haven't looked into that as yet.)

          Show
          cpoerschke Christine Poerschke added a comment - Patch with naive fix and test change. (There might be another approach i.e. avoiding docInfo being non-null at that place in that code path but I haven't looked into that as yet.)
          Hide
          cpoerschke Christine Poerschke added a comment -

          Preparatory patch to reduce code duplication in TestOriginalScoreFeature code.

          Show
          cpoerschke Christine Poerschke added a comment - Preparatory patch to reduce code duplication in TestOriginalScoreFeature code.
          Hide
          cpoerschke Christine Poerschke added a comment -

          Attaching partial extension to TestOriginalScoreFeature test.

          Show
          cpoerschke Christine Poerschke added a comment - Attaching partial extension to TestOriginalScoreFeature test.
          Hide
          zaky.vit Vitezslav Zak added a comment - - edited

          Hi Christine,

          I try run test scenario as you said. I run example techproducts. I add features and model by curl commands.

          curl -XPUT 'http://localhost:8983/solr/techproducts/schema/feature-store' --data-binary "@/home/zaky/prac/myFeatures.json" -H 'Content-type:application/json'
          curl -XPUT 'http://localhost:8983/solr/techproducts/schema/model-store' --data-binary "@/home/zaky/prac/myModel.json" -H 'Content-type:application/json'
          

          Features:

          [
            {
              "name" : "originalScore",
              "class" : "org.apache.solr.ltr.feature.OriginalScoreFeature",
              "params" : {}
            }
          ]
          

          Model:

          {
            "class" : "org.apache.solr.ltr.model.LinearModel",
            "name" : "myModel",
            "features" : [
              { "name" : "originalScore" }
            ],
            "params" : {
              "weights" : {
                "originalScore" : 1.0
              }
            }
          }
          

          Then I go to the Solr UI/core/Query - run a query with debugQuery And at the end of results, there is a error.

          I ran this url too and there is at the end error too:
          http://localhost:8983/solr/techproducts/select?debugQuery=on&indent=on&q=*:*&rq=

          {!ltr%20model=myModel%20reRankDocs=100}

          &wt=json

          Show
          zaky.vit Vitezslav Zak added a comment - - edited Hi Christine, I try run test scenario as you said. I run example techproducts. I add features and model by curl commands. curl -XPUT 'http: //localhost:8983/solr/techproducts/schema/feature-store' --data-binary "@/home/zaky/prac/myFeatures.json" -H 'Content-type:application/json' curl -XPUT 'http: //localhost:8983/solr/techproducts/schema/model-store' --data-binary "@/home/zaky/prac/myModel.json" -H 'Content-type:application/json' Features: [ { "name" : "originalScore" , "class" : "org.apache.solr.ltr.feature.OriginalScoreFeature" , "params" : {} } ] Model: { "class" : "org.apache.solr.ltr.model.LinearModel" , "name" : "myModel" , "features" : [ { "name" : "originalScore" } ], "params" : { "weights" : { "originalScore" : 1.0 } } } Then I go to the Solr UI/core/Query - run a query with debugQuery And at the end of results, there is a error. I ran this url too and there is at the end error too: http://localhost:8983/solr/techproducts/select?debugQuery=on&indent=on&q=*:*&rq= {!ltr%20model=myModel%20reRankDocs=100} &wt=json
          Hide
          cpoerschke Christine Poerschke added a comment -

          Hi Vitezslav Zak - thanks for this report.

          ... if I want to debug query in SOLR UI ...

          Could you outline the exact steps taken in the UI that result in the NPE being hit? Either via steps in the UI or as a curl or url request directly.

          Also, if the documentRecency and/or niceness features are removed, does the NPE occur then also? These two features use the initial_release_date and niceness fields which appear specific to your schema, reducing the example to a minimum would help reproduce the issue with (say) the techproducts example that ships with Solr and/or to create a test case for the scenario.

          Show
          cpoerschke Christine Poerschke added a comment - Hi Vitezslav Zak - thanks for this report. ... if I want to debug query in SOLR UI ... Could you outline the exact steps taken in the UI that result in the NPE being hit? Either via steps in the UI or as a curl or url request directly. Also, if the documentRecency and/or niceness features are removed, does the NPE occur then also? These two features use the initial_release_date and niceness fields which appear specific to your schema, reducing the example to a minimum would help reproduce the issue with (say) the techproducts example that ships with Solr and/or to create a test case for the scenario.

            People

            • Assignee:
              cpoerschke Christine Poerschke
              Reporter:
              zaky.vit Vitezslav Zak
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 0.5h
                0.5h
                Remaining:
                Remaining Estimate - 0.5h
                0.5h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development