[HADOOP-1220] block not found in blockMap

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Assignee: Dhruba Borthakur
• Reporter: Koji Noguchi
• Created: 06/Apr/07 22:03
• Updated: 08/Jul/09 16:42
• Resolved: 03/Oct/08 17:18

Description

From Hadoop-973 debug message, we had datanode constantly printing out the following message.

...
2007-04-02 23:59:50,122 WARN org.apache.hadoop.dfs.DataNode: java.io.IOException: Unexpected error trying to delete block blk_-3400783150525166031. Block not found in blockMap.
at org.apache.hadoop.dfs.FSDataset.invalidate(FSDataset.java:596)
at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460)
at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1053)
at java.lang.Thread.run(Thread.java:619)

2007-04-02 23:59:50,433 INFO org.apache.hadoop.dfs.DataNode: Served block blk_-8672111663356339464 to /72.30.127.164
2007-04-02 23:59:52,993 WARN org.apache.hadoop.dfs.DataNode: java.io.IOException: Unexpected error trying to delete block blk_-3400783150525166031. Block not found in blockMap.
at org.apache.hadoop.dfs.FSDataset.invalidate(FSDataset.java:596)
at org.apache.hadoop.dfs.DataNode.offerService(DataNode.java:460)
at org.apache.hadoop.dfs.DataNode.run(DataNode.java:1053)
at java.lang.Thread.run(Thread.java:619)
....

Ten minutes before these logs, there was
2007-04-02 20:42:49,648 INFO org.apache.hadoop.dfs.DataNode: Deleting block blk_-3400783150525166031

There was a file for that block in the directory.
/../../hadoop/dfs/data/data/subdir17/blk_-3400783150525166031

Issue Links

• is blocked by: HADOOP-1297 datanode sending block reports to namenode once every second