Hadoop Common
  1. Hadoop Common
  2. HADOOP-8833

fs -text should make sure to call inputstream.seek(0) before using input stream

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha
    • Component/s: fs
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      From Muddy Dixon on HADOOP-8449:

      Hi
      We found the changes in order of switch and guard block in

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException
      

      Because of this change, return value of

      codec.createInputStream(i)
      

      is changed if codec exists.

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          // check codecs
          CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
          CompressionCodec codec = cf.getCodec(p);
          if (codec != null) {
            return codec.createInputStream(i);
          }
      
          switch(i.readShort()) {
             // cases
          }
      

      New:

      private InputStream forMagic(Path p, FileSystem srcFs) throws IOException {
          FSDataInputStream i = srcFs.open(p);
      
          switch(i.readShort()) { // <=== index (or pointer) processes!!
            // cases
            default: {
              // Check the type of compression instead, depending on Codec class's
              // own detection methods, based on the provided path.
              CompressionCodecFactory cf = new CompressionCodecFactory(getConf());
              CompressionCodec codec = cf.getCodec(p);
              if (codec != null) {
                return codec.createInputStream(i);
              }
              break;
            }
          }
      
          // File is non-compressed, or not a file container we know.
          i.seek(0);
          return i;
        }
      

      Fix is to use i.seek(0) before we use i anywhere. I missed that.

      1. HADOOP-8833.patch
        4 kB
        Harsh J
      2. HADOOP-8833.patch
        4 kB
        Tom White
      3. HADOOP-8833.patch
        4 kB
        Karthik Kambatla

        Issue Links

          Activity

          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Harsh J made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Target Version/s 3.0.0, 2.0.3-alpha [ 12320357, 12323273 ]
          Fix Version/s 2.0.3-alpha [ 12323273 ]
          Resolution Fixed [ 1 ]
          Harsh J made changes -
          Target Version/s 3.0.0 [ 12320357 ] 3.0.0, 2.0.3-alpha [ 12320357, 12323273 ]
          Harsh J made changes -
          Target Version/s 2.0.2-alpha [ 12322473 ] 3.0.0 [ 12320357 ]
          Karthik Kambatla (Inactive) made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Karthik Kambatla (Inactive) made changes -
          Attachment HADOOP-8833.patch [ 12546074 ]
          Karthik Kambatla (Inactive) made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Karthik Kambatla (Inactive) made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Karthik Kambatla (Inactive) made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Tom White made changes -
          Attachment HADOOP-8833.patch [ 12546031 ]
          Harsh J made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 2.0.2-alpha [ 12322473 ]
          Harsh J made changes -
          Attachment HADOOP-8833.patch [ 12546011 ]
          Harsh J made changes -
          Field Original Value New Value
          Link This issue is broken by HADOOP-8449 [ HADOOP-8449 ]
          Harsh J created issue -

            People

            • Assignee:
              Harsh J
              Reporter:
              Harsh J
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development