[HADOOP-698] When DFS client fails to read from a datanode, the failed datanode is not excluded from target reselection

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Assignee: Milind Barve
• Reporter: Hairong Kuang
• Created: 08/Nov/06 23:03
• Updated: 08/Jul/09 16:42
• Resolved: 28/Nov/06 23:51

Description

In the method read(byte buf[ ], int off, int len) of DFSInputStream, when read fails, it calls "blockSeekTo" to reselect a datanode. However, the failed datanode does not feed back to blockSeekTo. The datanode selection algorithm works as follows:

• If the machine that the client is running on has a local copy, return the local machine;
• Otherwise, randomly pick up one location.

When the failed data node info does not feed back to target reselection, this leads to two flaws:
1. When a client fails to read from the local copy, for example, because of the checksum error, the local machine will always be chosen in retries.
2. Random selection may still return the same failed node.