Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9114

NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 5.0, 6.0, 6.1, 6.2, 6.3, 6.4
    • 6.5, 7.0
    • None
    • None

    Description

      Hello,

      I am getting a NPE when using the TermVectorComponent in combinition with ExactStatsCache.
      I am using SOLR 6.0.0 with 4 shards in total.

      This Bug is a duplicate of SOLR-8459
      It was already fixed in SOLR-8459 for SOLR 5.x but it is still open in the new SOLR 6.0.0
      Can you please fix it for the nes SOLR 6.0.0 as well? I already tried the patch of the 5.x bugfix on the SOLR 6.0.0 but the bug is still present.

      I set up my solrconfig.xml as described in these 2 links:
      TermVectorComponent:
      https://cwiki.apache.org/confluence/display/solr/The+Term+Vector+Component

      ExactStatsCache:
      https://cwiki.apache.org/confluence/display/solr/Distributed+Requests#Configuring+statsCache+implementation

      My snippets from solrconfig.xml:

      ...
        <!--
          Stats Cache
          https://cwiki.apache.org/confluence/display/solr/Distributed+Requests#Configuring+statsCache+implementation
        -->
        <statsCache class="org.apache.solr.search.stats.ExactStatsCache"/>
        <!-- 
          Term Vector Component
          https://cwiki.apache.org/confluence/display/solr/The+Term+Vector+Component
          -->
        <searchComponent name="tvComponent" class="org.apache.solr.handler.component.TermVectorComponent"/>
        <requestHandler name="/tvrh" class="org.apache.solr.handler.component.SearchHandler">
          <lst name="defaults">
            <bool name="tv">true</bool>
          </lst>
          <arr name="last-components">
            <str>tvComponent</str>
          </arr>
        </requestHandler>
      ...
      

      Unfortunately a request to SOLR like "http://host/solr/corename/tvrh?q=site_url_id:74" ends up with this NPE:

      69730 ERROR (qtp110456297-14) [c:SingleDomainSite_28 s:shard1 r:core_node1 x:SingleDomainSite_28_shard1_replica1] o.a.s.s.HttpSolrCall null:java.lang.NullPointerException
              at org.apache.solr.handler.component.TermVectorComponent.finishStage(TermVectorComponent.java:451)
              at org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:426)
              at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:155)
              at org.apache.solr.core.SolrCore.execute(SolrCore.java:2033)
              at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:652)
              at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:460)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
              at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
              at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
              at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
              at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
              at org.eclipse.jetty.server.Server.handle(Server.java:518)
              at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
              at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
              at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
              at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
              at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
              at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
              at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
              at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
              at java.lang.Thread.run(Thread.java:745)
      

      According to https://issues.apache.org/jira/browse/SOLR-7756 this Bug should be fixed with SOLR 5.3.0, but obviously this NPE is still present.
      Can you please help me here?

      Attachments

        1. SOLR-9914.patch
          5 kB
          Cao Manh Dat
        2. SOLR-9114.patch
          6 kB
          Cao Manh Dat

        Issue Links

          Activity

            caomanhdat Cao Manh Dat added a comment - - edited

            Updated path for this issue against lastest source.

            In this patch, I just set solr.StatsCache for TermVectorComponentDistributedTest, DistributedMLTComponentTest individually, because BaseDistributedSearchTestCase is too general for this tests.

            varun This issue has been sneaked in Solr source for a long time. We definitely should take an action soon.

            caomanhdat Cao Manh Dat added a comment - - edited Updated path for this issue against lastest source. In this patch, I just set solr.StatsCache for TermVectorComponentDistributedTest, DistributedMLTComponentTest individually, because BaseDistributedSearchTestCase is too general for this tests. varun This issue has been sneaked in Solr source for a long time. We definitely should take an action soon.
            caomanhdat Cao Manh Dat added a comment - - edited

            varun Hi Varun, I would like to commit the patch soon, unless you have a down vote for the patch.

            caomanhdat Cao Manh Dat added a comment - - edited varun Hi Varun, I would like to commit the patch soon, unless you have a down vote for the patch.
            varun Varun Thacker added a comment -

            Patch looks good!

            I think we need to modify MLTC#modifyStage with similar checks under MLTC#finishStage

            Also maybe lets move the choosing of stats cache to a utility method? Several test cases now do the same thing to pick the stats cache:
            TestDistribIDF
            TestExactStatsCache
            DistributedMLTComponentTest
            TermVectorComponentDistributedTest

            We'll need to remove unused imports before committing.

            varun Varun Thacker added a comment - Patch looks good! I think we need to modify MLTC#modifyStage with similar checks under MLTC#finishStage Also maybe lets move the choosing of stats cache to a utility method? Several test cases now do the same thing to pick the stats cache: TestDistribIDF TestExactStatsCache DistributedMLTComponentTest TermVectorComponentDistributedTest We'll need to remove unused imports before committing.
            caomanhdat Cao Manh Dat added a comment - - edited

            Updated patch for this issue, MLTC#modifyRequest() will make MLTC ignore all shardRequest, which

            purpose != PURPOSE_GET_MLT_RESULTS && purpose != PURPOSE_GET_TOP_IDS
            

            varun I think we can create an utility method later.

            caomanhdat Cao Manh Dat added a comment - - edited Updated patch for this issue, MLTC#modifyRequest() will make MLTC ignore all shardRequest, which purpose != PURPOSE_GET_MLT_RESULTS && purpose != PURPOSE_GET_TOP_IDS varun I think we can create an utility method later.
            varun Varun Thacker added a comment -

            +1 for the patch to commit. Test fails without the fix

            varun Varun Thacker added a comment - +1 for the patch to commit. Test fails without the fix

            Commit d34f549df6cee0db7cbff1ec2639262c2f7e21e2 in lucene-solr's branch refs/heads/master from caomanhdat
            [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d34f549 ]

            SOLR-9114: NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache

            jira-bot ASF subversion and git services added a comment - Commit d34f549df6cee0db7cbff1ec2639262c2f7e21e2 in lucene-solr's branch refs/heads/master from caomanhdat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d34f549 ] SOLR-9114 : NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache

            Commit 7852b20cd6951bb945a1faea73a888369098baf7 in lucene-solr's branch refs/heads/branch_6x from caomanhdat
            [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7852b20 ]

            SOLR-9114: NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache

            (cherry picked from commit d34f549df6cee0db7cbff1ec2639262c2f7e21e2)

            jira-bot ASF subversion and git services added a comment - Commit 7852b20cd6951bb945a1faea73a888369098baf7 in lucene-solr's branch refs/heads/branch_6x from caomanhdat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7852b20 ] SOLR-9114 : NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache (cherry picked from commit d34f549df6cee0db7cbff1ec2639262c2f7e21e2)
            caomanhdat Cao Manh Dat added a comment - - edited

            Thanks varun, adaffner!

            caomanhdat Cao Manh Dat added a comment - - edited Thanks varun , adaffner !

            Commit d34f549df6cee0db7cbff1ec2639262c2f7e21e2 in lucene-solr's branch refs/heads/apiv2 from caomanhdat
            [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d34f549 ]

            SOLR-9114: NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache

            jira-bot ASF subversion and git services added a comment - Commit d34f549df6cee0db7cbff1ec2639262c2f7e21e2 in lucene-solr's branch refs/heads/apiv2 from caomanhdat [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=d34f549 ] SOLR-9114 : NPE using TermVectorComponent, MoreLikeThisComponent in combination with ExactStatsCache

            People

              caomanhdat Cao Manh Dat
              adaffner Andreas Daffner
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: