[HDFS-2691] HA: Tests and fixes for pipeline targets and replica recovery

• Type: Sub-task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: HA branch (HDFS-1623)
• Fix Version/s: HA branch (HDFS-1623)
• Component/s: ha
• Target Version/s: HA branch (HDFS-1623)
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 15/Dec/11 22:43
• Updated: 11/Sep/12 03:37
• Resolved: 30/Jan/12 19:16

Description

Currently there are some TODOs around pipeline/recovery code in the HA branch. For example, commitBlockSynchronization only gets sent to the active NN which may have failed over by that point. So, we need to write some tests here and figure out what the correct behavior is.

Another related area is the treatment of targets in the pipeline. When a pipeline is created, the active NN adds the "expected locations" to the BlockInfoUnderConstruction, but the DN identifiers aren't logged with the OP_ADD. So after a failover, the BlockInfoUnderConstruction will have no targets and I imagine replica recovery would probably trigger some issues.

Issue Links

• is duplicated by: HDFS-2779 HA: Add lease recovery handling to HA
• relates to: HDFS-2791 If block report races with closing of file, replica is incorrectly marked corrupt