[HDFS-4796] Port HDFS-4721 'Speed up lease/block recovery when DN fails and a block goes into recovery' to branch 1

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Assignee: Unassigned
• Reporter: Ted Yu
• Created: 04/May/13 20:44
• Updated: 15/Oct/17 04:31
• Resolved: 15/Oct/17 04:31

Description

This was observed while doing HBase WAL recovery. HBase uses append to write to its write ahead log. So initially the pipeline is setup as

DN1 --> DN2 --> DN3

This WAL needs to be read when DN1 fails since it houses the HBase regionserver for the WAL.

HBase first recovers the lease on the WAL file. During recovery, we choose DN1 as the primary DN to do the recovery even though DN1 has failed and is not heartbeating any more.

To speedup lease/block recovery, we always choose the datanode with the most recent heartbeat.

Issue Links

• relates to: HDFS-4721 Speed up lease/block recovery when DN fails and a block goes into recovery