[HDFS-1984] 1073: Enable multiple checkpointers to run simultaneously

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Edit log branch (HDFS-1073)
• Fix Version/s: Edit log branch (HDFS-1073)
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Todd Lipcon
• Reporter: Todd Lipcon
• Created: 23/May/11 22:26
• Updated: 02/May/13 02:29
• Resolved: 25/May/11 01:03

Description

One of the motivations of HDFS-1073 is that it decouples the checkpoint process so that multiple checkpoints could be taken at the same time and not interfere with each other.

Currently on the 1073 branch this doesn't quite work right, since we have some state and validation in FSImage that's tied to a single fsimage_N – thus if two 2NNs perform a checkpoint at different transaction IDs, only one will succeed.

As a stress test, we can run two 2NNs each configured with the fs.checkpoint.interval set to "0" which causes them to continuously checkpoint as fast as they can.

Issue Links

• is depended upon by: HDFS-1991 HDFS-1073: Some refactoring of 2NN to easier share code with BN and CN