[SOLR-8119] Detect index corruption for all files on replication

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 5.5, 6.0
• Component/s: replication (java)
• Labels: difficulty-medium impact-high
• Assignee: Unassigned
• Reporter: Shalin Shekhar Mangar
• Created: 04/Oct/15 05:37
• Updated: 09/May/16 18:58

Description

Lucene writes checksums for large files but they aren't verified until a merge is necessary because it'd be too costly to go through the entire bytes. Only truncation of such files is checked during open. However, index replication is one activity that has to go through the entire file anyway so we can be more aggressive than Lucene in validating the checksum.

I propose that we validate all files, large and small, during replication.