[HADOOP-159] name node at 100% cpu, making redundant replications

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 0.1.1
• Fix Version/s: 0.2.0
• Assignee: Sameer Paranjpye
• Reporter: Yoram Arnon
• Created: 22/Apr/06 05:56
• Updated: 08/Jul/09 16:41
• Resolved: 31/May/06 02:04

Description

some hours after adding some new nodes to the cluster, the name node went into a state where it's consuming 100% cpu.
The log file keeps logging messages of the forms
060421 155049 Obsoleting block blk_8093115169359854355
060421 155049 Pending transfer (block blk_-6965677235456960523) from node1383:50010 to 2 destinations
060421 155049 Block report from node1283:50010: 2140 blocks.
060421 155049 Redundant addStoredBlock request received for block blk_-6836937139917042917 on node node1143:50010

many DFS operations time out, making useful work impossible.

restarting dfs solved the problem for a while, but it came back within an hour.