[HADOOP-1126] Optimize CPU usage when cluster restarts

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.12.1
• Assignee: Hairong Kuang
• Reporter: Dhruba Borthakur
• Created: 15/Mar/07 23:09
• Updated: 02/Feb/10 06:16
• Resolved: 17/Mar/07 20:19

Description

When the namenode restarts, all datanodes report their blocks to the namenode. The namenode inserts these blocks into neededReplication(). When safemode exists, pendingTransfers removes them from neededReplication. This is non-optimal because pendingTransfer consumes plenty of CPU just after exiting safe-mode.

The problem is that neededReplications.remove(block) does not remove a block if oldExpectedReplicas is exactly equal to oldReplicas.