Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6801

io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.0-alpha
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor.

      This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.
      Show
      Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor. This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.

      Description

      Following configuration keys in CommonConfigurationKeysPublic.java (former CommonConfigurationKeys.java):

      public static final String IO_SORT_MB_KEY = "io.sort.mb";
      public static final String IO_SORT_FACTOR_KEY = "io.sort.factor";

      are partially moved:

      • they were renamed to mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor respectively
      • they were moved to mapreduce project, documented in mapred-default.xml

      However:

      • they are still listed in CommonConfigurationKeysPublic.java as quoted above
      • strings "io.sort.mb" and "io.sort.factor" are used in SequenceFile.java in Hadoop Common project

      Not sure what the solution is, these constants should probably be removed from CommonConfigurationKeysPublic.java but I am not sure what's the best solution for SequenceFile.java.

        Attachments

        1. HADOOP-6801.r5.diff
          8 kB
          Harsh J
        2. HADOOP-6801.r4.diff
          8 kB
          Harsh J
        3. HADOOP-6801.r3.diff
          8 kB
          Harsh J
        4. HADOOP-6801.r2.diff
          4 kB
          Harsh J
        5. HADOOP-6801.r1.diff
          3 kB
          Harsh J
        6. HADOOP-6801.05.patch
          8 kB
          Allen Wittenauer

          Issue Links

            Activity

              People

              • Assignee:
                qwertymaniac Harsh J
                Reporter:
                zasran Erik Steffl
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: