[HDFS-2824] HA: failover does not succeed if prior NN died just after creating an edit log segment

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: HA branch (HDFS-1623)
• Fix Version/s: HA branch (HDFS-1623)
• Component/s: ha, namenode
• Target Version/s: HA branch (HDFS-1623)
• Hadoop Flags: Reviewed
• Assignee: Aaron Myers
• Reporter: Todd Lipcon
• Created: 21/Jan/12 22:58
• Updated: 02/Nov/12 23:06
• Resolved: 30/Jan/12 23:05

Description

In stress testing failover, I had the following failure:

• NN1 rolls edit logs and starts writing edits_inprogress_1000
• NN1 crashes before writing the START_LOG_SEGMENT transaction
• NN2 tries to become active, and calls recoverUnfinalizedSegment. Since the log file contains no valid transactions, it is marked as corrupt and renamed with the .corrupt suffix
• The sanity check in openLogsForWrite will refuse to open a new in-progress log at the same txid. Failover does not proceed.

Issue Links

• is related to: HDFS-3771 Namenode can't restart due to corrupt edit logs, timing issue with shutdown and edit log rolling