Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8833

fs -text should make sure to call inputstream.seek(0) before using input stream

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From Muddy Dixon on HADOOP-8449:

      Hi
      We found the changes in order of switch and guard block in

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException
      

      Because of this change, return value of

      codec.createInputStream(i)
      

      is changed if codec exists.

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          // check codecs
          CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
          CompressionCodec codec = cf.getCodec(p);
          if (codec != null) {
            return codec.createInputStream(i);
          }
      
          switch(i.readShort()) {
             // cases
          }
      

      New:

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          switch(i.readShort()) { // <=== index (or pointer) processes!!
            // cases
            default: {
              // Check the type of compression instead, depending on Codec class's
              // own detection methods, based on the provided path.
              CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
              CompressionCodec codec = cf.getCodec(p);
              if (codec != null) {
                return codec.createInputStream(i);
              }
              break;
            }
          }
      
          // File is non-compressed, or not a file container we know.
          i.seek(0);
          return i;
        }
      

      Fix is to use i.seek(0) before we use i anywhere. I missed that.

        Attachments

        1. HADOOP-8833.patch
          4 kB
          Karthik Kambatla
        2. HADOOP-8833.patch
          4 kB
          Thomas White
        3. HADOOP-8833.patch
          4 kB
          Harsh J

        Issue Links

          Activity

            People

            • Assignee:
              qwertymaniac Harsh J
              Reporter:
              qwertymaniac Harsh J

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment