Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8833

fs -text should make sure to call inputstream.seek(0) before using input stream

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.2-alpha
    • 2.0.3-alpha
    • fs
    • None
    • Reviewed

    Description

      From Muddy Dixon on HADOOP-8449:

      Hi
      We found the changes in order of switch and guard block in

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException
      

      Because of this change, return value of

      codec.createInputStream(i)
      

      is changed if codec exists.

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          // check codecs
          CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
          CompressionCodec codec = cf.getCodec(p);
          if (codec != null) {
            return codec.createInputStream(i);
          }
      
          switch(i.readShort()) {
             // cases
          }
      

      New:

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          switch(i.readShort()) { // <=== index (or pointer) processes!!
            // cases
            default: {
              // Check the type of compression instead, depending on Codec class's
              // own detection methods, based on the provided path.
              CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
              CompressionCodec codec = cf.getCodec(p);
              if (codec != null) {
                return codec.createInputStream(i);
              }
              break;
            }
          }
      
          // File is non-compressed, or not a file container we know.
          i.seek(0);
          return i;
        }
      

      Fix is to use i.seek(0) before we use i anywhere. I missed that.

      Attachments

        1. HADOOP-8833.patch
          4 kB
          Harsh J
        2. HADOOP-8833.patch
          4 kB
          Thomas White
        3. HADOOP-8833.patch
          4 kB
          Karthik Kambatla

        Issue Links

          Activity

            People

              qwertymaniac Harsh J
              qwertymaniac Harsh J
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: