Solr
  1. Solr
  2. SOLR-7866

getMaxVersionFromIndex is throwing an NPE if some segments are not current

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None
    • Flags:
      Important

      Description

      Reported by user on #solr irc (via Shalin)

      java.lang.NullPointerException
      	at org.apache.lucene.util.NumericUtils.prefixCodedToLong(NumericUtils.java:226)
      	at org.apache.lucene.util.NumericUtils.getMaxLong(NumericUtils.java:582)
      	at org.apache.solr.update.VersionInfo.getMaxVersionFromIndex(VersionInfo.java:234)
      	at org.apache.solr.update.UpdateLog.seedBucketsWithHighestVersion(UpdateLog.java:1580)
      	at org.apache.solr.update.UpdateLog.seedBucketsWithHighestVersion(UpdateLog.java:1610)
      	at org.apache.solr.core.SolrCore.seedVersionBuckets(SolrCore.java:861)
      	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:843)
      	at org.apache.solr.core.SolrCore.<init>(SolrCore.java:658)
      	at org.apache.solr.core.CoreContainer.create(CoreContainer.java:637)
      	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:381)
      	at org.apache.solr.core.CoreContainer$1.call(CoreContainer.java:375)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor$1.run(ExecutorUtil.java:148)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:745)
      
      1. SOLR-7866.patch
        0.8 kB
        Timothy Potter

        Issue Links

          Activity

          Hide
          Timothy Potter added a comment -

          At first, I thought this was due to 5.x code reading a 4.x index, but i was able to get the max version for an index created with 4.2 on branch_5x. So I don't know what's going on here exactly ...

          However, since this code is an optimization, I don't think we should fail to initialize the core, so I'm going to catch Exception instead of just IOException so that at least the index is usable, albeit with degraded indexing performance.

          Show
          Timothy Potter added a comment - At first, I thought this was due to 5.x code reading a 4.x index, but i was able to get the max version for an index created with 4.2 on branch_5x. So I don't know what's going on here exactly ... However, since this code is an optimization, I don't think we should fail to initialize the core, so I'm going to catch Exception instead of just IOException so that at least the index is usable, albeit with degraded indexing performance.
          Hide
          Hoss Man added a comment -

          i think the root cause here is LUCENE-6719.

          Show
          Hoss Man added a comment - i think the root cause here is LUCENE-6719 .
          Hide
          Timothy Potter added a comment -

          Patch to make sure there are terms before requesting the max.

          Show
          Timothy Potter added a comment - Patch to make sure there are terms before requesting the max.
          Hide
          Timothy Potter added a comment -

          Not sure how to create a unit test for this because even if terms is empty, i get a MAX value back:

             [junit4]   2> 61414 INFO  (qtp13679606-95) [n:127.0.0.1:55465_ c:c8n_vers_1x3 s:shard1 r:core_node1 x:c8n_vers_1x3_shard1_replica2] o.a.s.u.VersionInfo
             [junit4]   2>  versionTerms.size=-1
             [junit4]   2> 61417 INFO  (qtp13679606-95) [n:127.0.0.1:55465_ c:c8n_vers_1x3 s:shard1 r:core_node1 x:c8n_vers_1x3_shard1_replica2] o.a.s.u.VersionInfo Found MAX value 1508608304047194112 from Terms for _version_ in index
          
          Show
          Timothy Potter added a comment - Not sure how to create a unit test for this because even if terms is empty, i get a MAX value back: [junit4] 2> 61414 INFO (qtp13679606-95) [n:127.0.0.1:55465_ c:c8n_vers_1x3 s:shard1 r:core_node1 x:c8n_vers_1x3_shard1_replica2] o.a.s.u.VersionInfo [junit4] 2> versionTerms.size=-1 [junit4] 2> 61417 INFO (qtp13679606-95) [n:127.0.0.1:55465_ c:c8n_vers_1x3 s:shard1 r:core_node1 x:c8n_vers_1x3_shard1_replica2] o.a.s.u.VersionInfo Found MAX value 1508608304047194112 from Terms for _version_ in index
          Hide
          Timothy Potter added a comment -

          Dropping this ticket from Blocker for 5.3 to major status as I haven't been able to reproduce this issue. We could of course just code around this issue by catching the NPE but given that I can't reproduce it with empty / non-empty 4.x and 5.x indexes, I'm hesitant to change code in this area until LUCENE-6719 is resolved.

          Show
          Timothy Potter added a comment - Dropping this ticket from Blocker for 5.3 to major status as I haven't been able to reproduce this issue. We could of course just code around this issue by catching the NPE but given that I can't reproduce it with empty / non-empty 4.x and 5.x indexes, I'm hesitant to change code in this area until LUCENE-6719 is resolved.
          Hide
          ASF subversion and git services added a comment -

          Commit 1694385 from Timothy Potter in branch 'dev/trunk'
          [ https://svn.apache.org/r1694385 ]

          SOLR-7866: Harden code to prevent an unhandled NPE when trying to determine the max value of the version field.

          Show
          ASF subversion and git services added a comment - Commit 1694385 from Timothy Potter in branch 'dev/trunk' [ https://svn.apache.org/r1694385 ] SOLR-7866 : Harden code to prevent an unhandled NPE when trying to determine the max value of the version field.
          Hide
          ASF subversion and git services added a comment -

          Commit 1694388 from Timothy Potter in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1694388 ]

          SOLR-7866: Harden code to prevent an unhandled NPE when trying to determine the max value of the version field.

          Show
          ASF subversion and git services added a comment - Commit 1694388 from Timothy Potter in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694388 ] SOLR-7866 : Harden code to prevent an unhandled NPE when trying to determine the max value of the version field.
          Hide
          ASF subversion and git services added a comment -

          Commit 1694389 from Timothy Potter in branch 'dev/branches/lucene_solr_5_3'
          [ https://svn.apache.org/r1694389 ]

          SOLR-7866: Harden code to prevent an unhandled NPE when trying to determine the max value of the version field.

          Show
          ASF subversion and git services added a comment - Commit 1694389 from Timothy Potter in branch 'dev/branches/lucene_solr_5_3' [ https://svn.apache.org/r1694389 ] SOLR-7866 : Harden code to prevent an unhandled NPE when trying to determine the max value of the version field.
          Hide
          Shalin Shekhar Mangar added a comment -

          Bulk close for 5.3.0 release

          Show
          Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

            People

            • Assignee:
              Timothy Potter
              Reporter:
              Timothy Potter
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development