[HADOOP-855] HDFS should repair corrupted files

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.11.0
• Assignee: Wendy Chien
• Reporter: Wendy Chien
• Created: 04/Jan/07 00:36
• Updated: 08/Jul/09 16:42
• Resolved: 16/Jan/07 23:58

Description

While reading if we discover a mismatch between a block and checksum, we want to report this back to the namenode to delete the corrupted block or crc.

To implement this, we need to do the following:
DFSInputStream
1. move DFSInputStream out of DFSClient
2. add member variable to keep track of current datanode (the chosen node)

DistributedFileSystem
1. change reportChecksumFailure parameter crc from int to FSInputStream (needed to be able to delete it).
2. determine specific block and datanode from DFSInputStream passed to reportChecksumFailure
3. call namenode to delete block/crc vis DFSClient

ClientProtocol
1. add method to ask namenode to delete certain blocks on specifc datanode.

Namenode
1. add ability to delete certain blocks on specific datanode

Issue Links

• is related to: HADOOP-731 Sometimes when a dfs file is accessed and one copy has a checksum error the I/O command fails, even if another copy is alright.