Hadoop Common
  1. Hadoop Common
  2. HADOOP-2121

Unexpected IOException in DFSOutputStream.close()

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.14.3
    • Fix Version/s: 0.16.0
    • Component/s: None
    • Labels:
      None

      Description

      While running a test with datanodes with disk space limitations, Hairong noticed many IOExceptions like this :

      java.io.IOException: Mismatch in writeChunk() args
       at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.writeChunk(DFSClient.java:1575)
       at org.apache.hadoop.fs.FSOutputSummer.writeChecksumChunk(FSOutputSummer.java:140)
       at org.apache.hadoop.fs.FSOutputSummer.flushBuffer(FSOutputSummer.java:122)
       at org.apache.hadoop.dfs.DFSClient$DFSOutputStream.close(DFSClient.java:1715)
       at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:49)
       at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:64)
       at org.apache.hadoop.io.SequenceFile$Writer.close(SequenceFile.java:918)
       at org.apache.hadoop.mapred.SequenceFileOutputFormat$1.close(SequenceFileOutputFormat.java:72)
       at org.apache.hadoop.mapred.MapTask$DirectMapOutputCollector.close(MapTask.java:232)
       at org.apache.hadoop.mapred.MapTask.run(MapTask.java:197)
       at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1935)
      

      I will submit a patch. With the patch, we will still see an IOException, but an expected one.

      1. HADOOP-2121.patch
        2 kB
        Raghu Angadi
      2. HADOOP-2121.patch
        2 kB
        Raghu Angadi
      3. HADOOP-2121.patch
        0.5 kB
        Raghu Angadi

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Raghu Angadi
            Reporter:
            Raghu Angadi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development