[HADOOP-2121] Unexpected IOException in DFSOutputStream.close()

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.14.3
• Fix Version/s: 0.16.0
• Assignee: Raghu Angadi
• Reporter: Raghu Angadi
• Created: 29/Oct/07 22:32
• Updated: 08/Jul/09 16:42
• Resolved: 12/Nov/07 19:53

Description

While running a test with datanodes with disk space limitations, Hairong noticed many IOExceptions like this :

java.io.IOException: Mismatch in writeChunk() args
 at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:1575)
 at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:140)
 at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:122)
 at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1715)
 at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
 at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
 at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:918)
 at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
 at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.close(MapTask.java:232)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:197)
 at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1935)

I will submit a patch. With the patch, we will still see an IOException, but an expected one.