[HDFS-3335] check for edit log corruption at the end of the log

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0
• Fix Version/s: 2.0.2-alpha
• Hadoop Flags: Reviewed
• Assignee: Colin McCabe
• Reporter: Colin McCabe
• Created: 27/Apr/12 20:20
• Updated: 07/Sep/12 21:09
• Resolved: 15/May/12 00:44

Description

Even after encountering an OP_INVALID, we should check the end of the edit log to make sure that it contains no more edits.

This will catch things like rare race conditions or log corruptions that would otherwise remain undetected. They will got from being silent data loss scenarios to being cases that we can detect and fix.

Using recovery mode, we can choose to ignore the end of the log if necessary.

Issue Links

• relates to: HDFS-3479 backport HDFS-3335 (check for edit log corruption at the end of the log) to branch-1