[HADOOP-1497] Possibility of duplicate blockids if dead-datanodes come back up after corresponding files were deleted

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Assignee: Dhruba Borthakur
• Reporter: Dhruba Borthakur
• Created: 15/Jun/07 21:28
• Updated: 08/Jul/09 16:42
• Resolved: 14/May/08 07:10

Description

Suppose a datanode D has a block B that belongs to file F. Suppose the datanode D dies and the namenode replicates those blocks to other datanodes. No, suppose the user deletes file F. The namenode removes all the blocks that belonged to file F. Now, suppose a new file F1 is created and the namenode generates the same blockid B for this new file F1.

Suppose the old datanode D comes back to life. Now we have a valid corrupted block B on datanode D.

This case is possibly detected by the Client (using CRC). But does HDFS need to handle this scenario better?

Issue Links

• duplicates: HADOOP-158 dfs should allocate a random blockid range to a file, then assign ids sequentially to blocks in the file
• is related to: HADOOP-1700 Append to files in HDFS