Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8833

fs -text should make sure to call inputstream.seek(0) before using input stream

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From Muddy Dixon on HADOOP-8449:

      Hi
      We found the changes in order of switch and guard block in

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException
      

      Because of this change, return value of

      codec.createInputStream(i)
      

      is changed if codec exists.

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          // check codecs
          CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
          CompressionCodec codec = cf.getCodec(p);
          if (codec != null) {
            return codec.createInputStream(i);
          }
      
          switch(i.readShort()) {
             // cases
          }
      

      New:

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          switch(i.readShort()) { // <=== index (or pointer) processes!!
            // cases
            default: {
              // Check the type of compression instead, depending on Codec class's
              // own detection methods, based on the provided path.
              CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
              CompressionCodec codec = cf.getCodec(p);
              if (codec != null) {
                return codec.createInputStream(i);
              }
              break;
            }
          }
      
          // File is non-compressed, or not a file container we know.
          i.seek(0);
          return i;
        }
      

      Fix is to use i.seek(0) before we use i anywhere. I missed that.

        Attachments

        1. HADOOP-8833.patch
          4 kB
          Harsh J
        2. HADOOP-8833.patch
          4 kB
          Tom White
        3. HADOOP-8833.patch
          4 kB
          Karthik Kambatla

          Issue Links

            Activity

              People

              • Assignee:
                qwertymaniac Harsh J
                Reporter:
                qwertymaniac Harsh J
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: