Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6801

io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0-alpha
    • 2.9.0, 3.0.0-alpha4
    • None
    • None
    • Reviewed
    • Hide
      Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor.

      This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.
      Show
      Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor. This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.

    Description

      Following configuration keys in CommonConfigurationKeysPublic.java (former CommonConfigurationKeys.java):

      public static final String IO_SORT_MB_KEY = "io.sort.mb";
      public static final String IO_SORT_FACTOR_KEY = "io.sort.factor";

      are partially moved:

      • they were renamed to mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor respectively
      • they were moved to mapreduce project, documented in mapred-default.xml

      However:

      • they are still listed in CommonConfigurationKeysPublic.java as quoted above
      • strings "io.sort.mb" and "io.sort.factor" are used in SequenceFile.java in Hadoop Common project

      Not sure what the solution is, these constants should probably be removed from CommonConfigurationKeysPublic.java but I am not sure what's the best solution for SequenceFile.java.

      Attachments

        1. HADOOP-6801.05.patch
          8 kB
          Allen Wittenauer
        2. HADOOP-6801.r5.diff
          8 kB
          Harsh J
        3. HADOOP-6801.r4.diff
          8 kB
          Harsh J
        4. HADOOP-6801.r3.diff
          8 kB
          Harsh J
        5. HADOOP-6801.r2.diff
          4 kB
          Harsh J
        6. HADOOP-6801.r1.diff
          3 kB
          Harsh J

        Issue Links

          Activity

            People

              qwertymaniac Harsh J
              zasran Erik Steffl
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: