Hadoop Common
  1. Hadoop Common
  2. HADOOP-7870

fix SequenceFile#createWriter with boolean createParent arg to respect createParent.

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: 1.1.0, 0.23.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After HBASE-6840, one set of calls to createNonRecursive(...) seems fishy - the new boolean createParent variable from the signature isn't used at all.

      +  public static Writer
      +    createWriter(FileSystem fs, Configuration conf, Path name,
      +                 Class keyClass, Class valClass, int bufferSize,
      +                 short replication, long blockSize, boolean createParent,
      +                 CompressionType compressionType, CompressionCodec codec,
      +                 Metadata metadata) throws IOException {
      +    if ((codec instanceof GzipCodec) &&
      +        !NativeCodeLoader.isNativeCodeLoaded() &&
      +        !ZlibFactory.isNativeZlibLoaded(conf)) {
      +      throw new IllegalArgumentException("SequenceFile doesn't work with " +
      +                                         "GzipCodec without native-hadoop code!");
      +    }
      +
      +    switch (compressionType) {
      +    case NONE:
      +      return new Writer(conf, 
      +          fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
      +          keyClass, valClass, metadata).ownStream();
      +    case RECORD:
      +      return new RecordCompressWriter(conf, 
      +          fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
      +          keyClass, valClass, codec, metadata).ownStream();
      +    case BLOCK:
      +      return new BlockCompressWriter(conf,
      +          fs.createNonRecursive(name, true, bufferSize, replication, blockSize, null),
      +          keyClass, valClass, codec, metadata).ownStream();
      +    default:
      +      return null;
      +    }
      +  } 
      +
      

      Nicolas Spiegelberg suggests changing it to

       
      if (createParent) { use fs.create() } 
      else { use fs.createNonRecursive(); }
      
      1. hadoop-7870.patch
        3 kB
        Eli Collins
      2. hadoop-7870.patch
        3 kB
        Jonathan Hsieh
      3. hadoop-7870-1.patch
        2 kB
        Eli Collins

        Issue Links

          Activity

            People

            • Assignee:
              Jonathan Hsieh
              Reporter:
              Jonathan Hsieh
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development