[HDFS-2282] Semi-harmless race between block reports and block invalidation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.20.203.0, 2.0.0-alpha
• Component/s: datanode
• Assignee: Unassigned
• Reporter: Todd Lipcon
• Created: 24/Aug/11 02:52
• Updated: 10/Mar/15 04:35
• Resolved: 19/Mar/14 23:07

Description

In the 0.20.203 codebase, block reports are not synchronized in any way against mutations to the actual file structure on disk. If a file is removed from a directory while the block report is scanning that directory, it will be mistakenly reported as existing with a length of 0, since File.length() on a non-existent file returns 0.

This results in an error being logged on the DataNode when the NN sends it a second block deletion request for the already-deleted block. I believe it to be harmless, but the error message can concern users.

This was fixed in the 0.20 code line in HDFS-2379. This jira remains open to track the port to 0.24.

Issue Links

• is related to: HDFS-2379 0.20: Allow block reports to proceed without holding FSDataset lock