[HDFS-1371] One bad node can incorrectly flag many files as corrupt

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.1, 0.23.0
• Fix Version/s: 0.23.0
• Component/s: hdfs-client, namenode

Environment

yahoo internal version
[knoguchi@gwgd4003 ~]$ hadoop version
Hadoop 0.20.104.3.1007030707

• Hadoop Flags: Reviewed
• Assignee: Tanping Wang
• Reporter: Koji Noguchi
• Created: 03/Sep/10 16:10
• Updated: 21/May/11 12:38
• Resolved: 19/May/11 22:34

Description

On our cluster, 12 files were reported as corrupt by fsck even though the replicas on the datanodes were healthy.
Turns out that all the replicas (12 files x 3 replicas per file) were reported corrupt from one node.

Surprisingly, these files were still readable/accessible from dfsclient (-get/-cat) without any problems.