[HADOOP-8833] fs -text should make sure to call inputstream.seek(0) before using input stream

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.2-alpha
• Fix Version/s: 2.0.3-alpha
• Component/s: fs
• Hadoop Flags: Reviewed
• Assignee: Harsh J
• Reporter: Harsh J
• Created: 21/Sep/12 05:47
• Updated: 15/Feb/13 13:12
• Resolved: 22/Sep/12 20:10

Description

From Muddy Dixon on HADOOP-8449:

Hi
We found the changes in order of switch and guard block in

private InputStream forMagic(Path p, FileSystem srcFs) throws IOException

Because of this change, return value of

codec.createInputStream(i)

is changed if codec exists.

private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
    FSDataInputStream i = srcFs.open(p);

    // check codecs
    CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
    CompressionCodec codec = cf.getCodec(p);
    if (codec != null) {
      return codec.createInputStream(i);
    }

    switch(i.readShort()) {
       // cases
    }

New:

private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
    FSDataInputStream i = srcFs.open(p);

    switch(i.readShort()) { // <=== index (or pointer) processes!!
      // cases
      default: {
        // Check the type of compression instead, depending on Codec class's
        // own detection methods, based on the provided path.
        CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
        CompressionCodec codec = cf.getCodec(p);
        if (codec != null) {
          return codec.createInputStream(i);
        }
        break;
      }
    }

    // File is non-compressed, or not a file container we know.
    i.seek(0);
    return i;
  }

Fix is to use i.seek(0) before we use i anywhere. I missed that.

Issue Links

• is broken by: HADOOP-8449 hadoop fs -text fails with compressed sequence files with the codec file extension