[HADOOP-4597] Under-replicated blocks are not calculated if the name-node is forced out of safe-mode.

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.18.0
• Fix Version/s: 0.18.3
• Hadoop Flags: Reviewed
• Assignee: Konstantin Shvachko
• Reporter: Konstantin Shvachko
• Created: 06/Nov/08 00:07
• Updated: 08/Jul/09 16:43
• Resolved: 07/Nov/08 02:13

Description

Currently during name-node startup under-replicated blocks are not added to the neededReplications queue until the name-node leaves safe mode. This is an optimization since otherwise all blocks will first go into the under-replicated queue and then most of them will be removed from it.
When the name-node leaves safe-mode automatically it checks all blocks to have a correct number of replicas (processMisReplicatedBlocks()).
When the name-node leaves safe-mode manually it does not perform the checkup.
In the latter case all under-replicated blocks remain not replicated forever because there is no alternative mechanism to trigger replications.
The proposal is to call processMisReplicatedBlocks() any time the name-node leaves safe mode - automatically or manually.
In addition to solving that problem this could be an alternative mechanism for refreshing neededReplications and excessReplicateMap sets.

Issue Links

• is related to: HDFS-406 Automate test for Hadoop-4597
• relates to: HADOOP-4610 Always calculate mis-replicated blocks when safe-mode is turned off.