Solr
  1. Solr
  2. SOLR-7935

NPE from VersionInfo.lookupVersion during core reload

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.4
    • Component/s: None
    • Labels:
      None

      Description

      The test from SOLR-7836 sometimes fails with a NPE where an add looks up version info from the index.

      1. SOLR-7935.patch
        1 kB
        Yonik Seeley

        Activity

        Hide
        Yonik Seeley added a comment -

        Here's a trace from one of the fails

           > Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher
           >    at __randomizedtesting.SeedInfo.seed([FF8C00109137C2A5]:0)
           >    at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166)
           > Caused by: org.apache.solr.common.SolrException: Error opening new searcher
           >    at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1673)
           >    at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1530)
           >    at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:202)
           >    at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:783)
           >    at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:195)
           >    at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1088)
           >    at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:705)
           >    at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104)
           >    at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470)
           >    at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134)
           >    at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113)
           >    at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76)
           >    at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98)
           >    at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74)
           >    at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151)
           >    at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079)
           >    at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131)
           >    at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104)
           >    at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250)
           >    at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200)
           >    at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46)
           >    at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156)
           > Caused by: java.lang.NullPointerException
           >    at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642)
           >    ... 21 more
        
        Show
        Yonik Seeley added a comment - Here's a trace from one of the fails > Caused by: java.lang.RuntimeException: org.apache.solr.common.SolrException: Error opening new searcher > at __randomizedtesting.SeedInfo.seed([FF8C00109137C2A5]:0) > at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:166) > Caused by: org.apache.solr.common.SolrException: Error opening new searcher > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1673) > at org.apache.solr.core.SolrCore.getRealtimeSearcher(SolrCore.java:1530) > at org.apache.solr.update.VersionInfo.getVersionFromIndex(VersionInfo.java:202) > at org.apache.solr.update.UpdateLog.lookupVersion(UpdateLog.java:783) > at org.apache.solr.update.VersionInfo.lookupVersion(VersionInfo.java:195) > at org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:1088) > at org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:705) > at org.apache.solr.update.processor.LogUpdateProcessor.processAdd(LogUpdateProcessorFactory.java:104) > at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:470) > at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:134) > at org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:113) > at org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:76) > at org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:98) > at org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) > at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:151) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:2079) > at org.apache.solr.servlet.DirectSolrConnection.request(DirectSolrConnection.java:131) > at org.apache.solr.SolrTestCaseJ4.updateJ(SolrTestCaseJ4.java:1104) > at org.apache.solr.SolrTestCaseJ4.addAndGetVersion(SolrTestCaseJ4.java:1250) > at org.apache.solr.search.TestReloadDeadlock.addDoc(TestReloadDeadlock.java:200) > at org.apache.solr.search.TestReloadDeadlock.access$100(TestReloadDeadlock.java:46) > at org.apache.solr.search.TestReloadDeadlock$1.run(TestReloadDeadlock.java:156) > Caused by: java.lang.NullPointerException > at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1642) > ... 21 more
        Hide
        Yonik Seeley added a comment -

        OK, so what I think is happening here:

        • ulog is reused for the new core
        • ulog has a reference to the uhandler that is set at the end of the DUH2 constructor
        • a request that needs a version check comes in after the ulog.uhandler reference is set, but before the core.uhandler reference is set, leading to a NPE when it is used to try and open a new realtime searcher
        Show
        Yonik Seeley added a comment - OK, so what I think is happening here: ulog is reused for the new core ulog has a reference to the uhandler that is set at the end of the DUH2 constructor a request that needs a version check comes in after the ulog.uhandler reference is set, but before the core.uhandler reference is set, leading to a NPE when it is used to try and open a new realtime searcher
        Hide
        Yonik Seeley added a comment -

        Here's a simple patch that avoids going through the update handler to get the latest writer.

        Show
        Yonik Seeley added a comment - Here's a simple patch that avoids going through the update handler to get the latest writer.
        Hide
        Yonik Seeley added a comment -

        OK, looping the test overnight seems to verify that this patch works to fix the NPE fails.

        Show
        Yonik Seeley added a comment - OK, looping the test overnight seems to verify that this patch works to fix the NPE fails.
        Hide
        ASF subversion and git services added a comment -

        Commit 1696417 from Yonik Seeley in branch 'dev/trunk'
        [ https://svn.apache.org/r1696417 ]

        SOLR-7935:fix race that can cause a NPE in openNewSearcher called from an update concurrent with a core reload

        Show
        ASF subversion and git services added a comment - Commit 1696417 from Yonik Seeley in branch 'dev/trunk' [ https://svn.apache.org/r1696417 ] SOLR-7935 :fix race that can cause a NPE in openNewSearcher called from an update concurrent with a core reload
        Hide
        ASF subversion and git services added a comment -

        Commit 1696420 from Yonik Seeley in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1696420 ]

        SOLR-7935:fix race that can cause a NPE in openNewSearcher called from an update concurrent with a core reload

        Show
        ASF subversion and git services added a comment - Commit 1696420 from Yonik Seeley in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1696420 ] SOLR-7935 :fix race that can cause a NPE in openNewSearcher called from an update concurrent with a core reload

          People

          • Assignee:
            Yonik Seeley
            Reporter:
            Yonik Seeley
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development