[SOLR-17120] NullPointerException in UpdateLog.applyOlderUpdates in solr 6.6-9.4 involving partial updates

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 6.6.2, 8.11.2, 9.4
• Fix Version/s: 8.11.3, 9.5
• Component/s: update

Environment

The issue occurred on Linux, CentOS 7.9, with the following JDK version:

openjdk version "11.0.20" 2023-07-18 LTS
OpenJDK Runtime Environment (Red_Hat-11.0.20.0.8-1.el7_9) (build 11.0.20+8-LTS)
OpenJDK 64-Bit Server VM (Red_Hat-11.0.20.0.8-1.el7_9) (build 11.0.20+8-LTS, mixed mode, sharing)

• Assignee: Christine Poerschke
• Reporter: Calvin Smith
• Created: 18/Jan/24 01:55
• Updated: 09/Feb/24 16:38
• Resolved: 24/Jan/24 18:45

Description

I mailed the solr-users mailing list about this issue, but didn't get any responses there, so am creating this issue. The subject of the email thread for additional context was "NullPointerException in UpdateLog.applyOlderUpdates under solr 8&9 involving partial updates and high update load" - link: https://lists.apache.org/thread/n9zm4gocl7cf073syy1159dy6ojjrywl

I'm seeing a Solr HTTP 500 error when performing a partial update of a document that turns out to triggered by there having been a recent update of the same document that included a partial update that set a field to null. I've observed the behavior in versions 6.6.2, 8.11.2, and 9.4.0, which are the only 3 versions I've tried.

To give an example, an update doc like

 

{
    "id": "123", 
    "camera_unit": {"set": null}
}

 

followed shortly thereafter (not sure of exact timing, but I was using a commitWithin of 600s and the subsequent updates were less than 20 seconds later), after some other updates had happened for different documents, there was another update of the same document, like

 

{
    "id": "123", 
    "playlist": {
      "set": [
        12345
      ]
    },
    "playlist_index_321": {
      "set": 0
    }
}

 

This later update may, but doesn't always, cause the NullPointerException, so there is some other factor such as the state of the tlog that also has to be satisfied for the error to occur.

The exception is thrown by the following code in UpdateLog.java (org.apache.solr.update.UpdateLog):

 

/** Add all fields from olderDoc into newerDoc if not already present in newerDoc */
  private void applyOlderUpdates(
      SolrDocumentBase<?, ?> newerDoc, SolrInputDocument olderDoc, Set<String> mergeFields) {
    for (String fieldName : olderDoc.getFieldNames()) {
      // if the newerDoc has this field, then this field from olderDoc can be ignored
      if (!newerDoc.containsKey(fieldName)
          && (mergeFields == null || mergeFields.contains(fieldName))) {
        for (Object val : olderDoc.getFieldValues(fieldName)) {
          newerDoc.addField(fieldName, val);
        }
      }
    }
  }

 

The exception is due to the inner for statement trying to iterate over the null value being returned by olderDoc.getFieldValues(fieldName).

When I change that method to the following:

 

/** Add all fields from olderDoc into newerDoc if not already present in newerDoc */
  private void applyOlderUpdates(
      SolrDocumentBase<?, ?> newerDoc, SolrInputDocument olderDoc, Set<String> mergeFields) {
    for (String fieldName : olderDoc.getFieldNames()) {
      // if the newerDoc has this field, then this field from olderDoc can be ignored
      if (!newerDoc.containsKey(fieldName)
          && (mergeFields == null || mergeFields.contains(fieldName))) {
        Collection<Object> values = olderDoc.getFieldValues(fieldName);
        if (values == null) {
            newerDoc.addField(fieldName, null);
        } else {
            for (Object val : values) {
              newerDoc.addField(fieldName, val);
            }
        }
      }
    }
  }

 

Then after rebuilding the solr-core JAR with ./gradlew devFull and restarting Solr with that custom jar file, I can no longer reproduce the error.

I'm not familiar with the Solr codebase though and am not at all sure that newerDoc.addField(fieldName, null) is what should be done there.

Issue Links

• links to: GitHub Pull Request #2214
• links to: GitHub Pull Request #2683