[SOLR-6251] incorrect 'missing required field' during update - document definitely has it

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 4.8
• Component/s: SolrCloud
• Labels: replication

Environment

4.8.0. Two nodes, SolrCloud, external ZK ensemble. All on EC2. The two hosts are round-robin'd behind an ELB.

• Assignee: Unassigned
• Reporter: Nathan Neulinger
• Created: 15/Jul/14 22:46
• Updated: 18/Jul/14 02:37
• Resolved: 15/Jul/14 22:57

Description

Document added on solr1. We can see the distribute take place from solr1 to solr2 and returning a success. Subsequent searches returning document, clearly showing the field as being there. Later on, an update is done to add to an element of the document - and the update fails. The update was sent to solr2 instance.

Schema marks the 'timestamp' field as required, so the initial insert should not work if the field isn't present.

Symptom is intermittent - we're seeing this randomly, with no warning or triggering that we can see, but in all cases, it's getting the error in response to an update when the instance tries to distribute the change to the other node.

Searches that were run AFTER the update also show the field as being present in the document.

Will add full trace of operations in the comments shortly. pcap captures of ALL traffic for the two nodes on 8983 is available if requested.

Issue Links

• is superceded by: SOLR-6255 Misleading error message when usable questionable update syntax