Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6801

io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0-alpha
    • 2.9.0, 3.0.0-alpha4
    • None
    • None
    • Reviewed
    • Hide
      Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor.

      This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.
      Show
      Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor. This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.

    Description

      Following configuration keys in CommonConfigurationKeysPublic.java (former CommonConfigurationKeys.java):

      public static final String IO_SORT_MB_KEY = "io.sort.mb";
      public static final String IO_SORT_FACTOR_KEY = "io.sort.factor";

      are partially moved:

      • they were renamed to mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor respectively
      • they were moved to mapreduce project, documented in mapred-default.xml

      However:

      • they are still listed in CommonConfigurationKeysPublic.java as quoted above
      • strings "io.sort.mb" and "io.sort.factor" are used in SequenceFile.java in Hadoop Common project

      Not sure what the solution is, these constants should probably be removed from CommonConfigurationKeysPublic.java but I am not sure what's the best solution for SequenceFile.java.

      Attachments

        1. HADOOP-6801.r1.diff
          3 kB
          Harsh J
        2. HADOOP-6801.r2.diff
          4 kB
          Harsh J
        3. HADOOP-6801.r3.diff
          8 kB
          Harsh J
        4. HADOOP-6801.r4.diff
          8 kB
          Harsh J
        5. HADOOP-6801.r5.diff
          8 kB
          Harsh J
        6. HADOOP-6801.05.patch
          8 kB
          Allen Wittenauer

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            qwertymaniac Harsh J
            zasran Erik Steffl
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment