[HDFS-855] namenode can save images in parallel to all directories in fs.name.dir

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: namenode
• Assignee: Unassigned
• Reporter: Dhruba Borthakur
• Created: 27/Dec/09 08:16
• Updated: 30/Sep/10 23:43
• Resolved: 30/Sep/10 23:43

Description

The namenode restart times can be reduced if the namenode can save its image to multiple directories (specified in fs.name.dir) in parallel.

The NN has a 6 GB fsimage and 1 MB edits file. The NN needed 10 minutes to load fsimage/edits into memory. It needs 7 minutes to read the 6 GB image. There are two directories in fs.name.dir. it takes about 2 minute to save the image into first directory and another 2 minutes to save the image to the second directory.

Issue Links

• duplicates: HDFS-1071 savenamespace should write the fsimage to all configured fs.name.dir in parallel
• is related to: HADOOP-3022 Fast Cluster Restart