[HADOOP-3685] Unbalanced replication target

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.17.0
• Fix Version/s: 0.17.2
• Hadoop Flags: Reviewed
• Assignee: Hairong Kuang
• Reporter: Koji Noguchi
• Created: 02/Jul/08 22:14
• Updated: 08/Jul/09 16:43
• Resolved: 08/Jul/08 23:52

Description

In HADOOP-3633, namenode was assigning some datanodes to receive hundreds of blocks in a short period which caused datanodes to go out of memroy(threads).
Most of them were from remote rack.

Looking at the code,

    166           chooseLocalRack(results.get(1), excludedNodes, blocksize,
    167                           maxNodesPerRack, results);

was sometimes not choosing the local rack of the writer(source).

As a result, when a datanode goes down, other datanodes on the same rack were getting large number of blocks from remote racks.

Issue Links

• relates to: HADOOP-3633 Uncaught exception in DataXceiveServer