Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Invalid
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Many reduce tasks got killed due to checksum error. The strange thing is that the file was generated by the sort function, and was on a local disk. Here is the stack:

      Checksum error: ../task_0011_r_000140_0/all.2.1 at 5342920704
      at org.apache.hadoop.fs.FSDataInputStream$Checker.verifySum(FSDataInputStream.java:134)
      at org.apache.hadoop.fs.FSDataInputStream$Checker.read(FSDataInputStream.java:110)
      at org.apache.hadoop.fs.FSDataInputStream$PositionCache.read(FSDataInputStream.java:170)
      at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
      at java.io.BufferedInputStream.read1(BufferedInputStream.java:256)
      at java.io.BufferedInputStream.read(BufferedInputStream.java:313)
      at java.io.DataInputStream.readFully(DataInputStream.java:176)
      at org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:55)
      at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:89)
      at org.apache.hadoop.io.SequenceFile$Reader.readBuffer(SequenceFile.java:1061)
      at org.apache.hadoop.io.SequenceFile$Reader.seekToCurrentValue(SequenceFile.java:1126)
      at org.apache.hadoop.io.SequenceFile$Reader.nextRaw(SequenceFile.java:1354)
      at org.apache.hadoop.io.SequenceFile$Sorter$MergeStream.next(SequenceFile.java:1880)
      at org.apache.hadoop.io.SequenceFile$Sorter$MergeQueue.merge(SequenceFile.java:1938)
      at org.apache.hadoop.io.SequenceFile$Sorter$MergePass.run(SequenceFile.java:1802)
      at org.apache.hadoop.io.SequenceFile$Sorter.mergePass(SequenceFile.java:1749)
      at org.apache.hadoop.io.SequenceFile$Sorter.sort(SequenceFile.java:1494)
      at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:240)
      at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1066)

        Attachments

          Activity

            People

            • Assignee:
              owen.omalley Owen O'Malley
              Reporter:
              runping Runping Qi
            • Votes:
              2 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: