[LUCENE-5842] Validate checksum footers for postings lists, docvalues, storedfields, termvectors on init

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 4.10, 6.0
• Lucene Fields: New
• Assignee: Unassigned
• Reporter: Robert Muir
• Created: 23/Jul/14 12:04
• Updated: 28/Aug/22 14:12
• Resolved: 23/Jul/14 15:30

Description

For small files (e.g. where we read in all the bytes anyway), we currently validate the checksum on reader init.

But for larger files like .doc/.frq/.pos/.dvd/.fdt/.tvd we currently do nothing at all on init, as it would be too expensive.

We should at least do this:

// NOTE: data file is too costly to verify checksum against all the bytes on 
// open, but for now we at least verify proper structure of the checksum 
// footer: which looks for FOOTER_MAGIC + algorithmID. This is cheap 
// and can detect some forms of corruption such as file truncation.
CodecUtil.retrieveChecksum(data);