Hadoop Common
  1. Hadoop Common
  2. HADOOP-6663

BlockDecompressorStream get EOF exception when decompressing the file compressed from empty file

    Details

    • Hadoop Flags:
      Reviewed
    • Release Note:
      Fix EOF exception in BlockDecompressorStream when decompressing previous compressed empty file
    • Tags:
      hadoop io compress BlockDecompressorStream

      Description

      An empty file can be compressed using BlockDecompressorStream, which is for block-based compressiong algorithm such as LZO. However, when decompressing the compressed file, BlockDecompressorStream get EOF exception.

      Here is a typical exception stack:

      java.io.EOFException
      at org.apache.hadoop.io.compress.BlockDecompressorStream.rawReadInt(BlockDecompressorStream.java:125)
      at org.apache.hadoop.io.compress.BlockDecompressorStream.getCompressedData(BlockDecompressorStream.java:96)
      at org.apache.hadoop.io.compress.BlockDecompressorStream.decompress(BlockDecompressorStream.java:82)
      at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:74)
      at java.io.InputStream.read(InputStream.java:85)
      at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)
      at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:134)
      at org.apache.hadoop.mapred.LineRecordReader.next(LineRecordReader.java:39)
      at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.moveToNext(MapTask.java:186)
      at org.apache.hadoop.mapred.MapTask$TrackedRecordReader.next(MapTask.java:170)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:48)
      at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:18)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
      at org.apache.hadoop.mapred.Child.main(Child.java:196)

      1. HADOOP-6663-0.20.2.patch
        0.6 kB
        Bennie Schut
      2. HADOOP-6663.patch
        6 kB
        Tom White
      3. BlockDecompressorStream.java.patch
        6 kB
        Kang Xiao
      4. BlockDecompressorStream.java.patch
        6 kB
        Kang Xiao
      5. BlockDecompressorStream.patch
        0.6 kB
        Kang Xiao

        Activity

          People

          • Assignee:
            Kang Xiao
            Reporter:
            Kang Xiao
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development