[HADOOP-6801] io.sort.mb and io.sort.factor were renamed and moved to mapreduce but are still in CommonConfigurationKeysPublic.java and used in SequenceFile.java

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0-alpha
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Hadoop Flags: Reviewed

Release Note

Two new configuration keys, seq.io.sort.mb and seq.io.sort.factor have been introduced for the SequenceFile's Sorter feature to replace older, deprecated property keys of io.sort.mb and io.sort.factor.

This only affects direct users of the org.apache.hadoop.io.SequenceFile.Sorter Java class. For controlling MR2's internal sorting instead, use the existing config keys of mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor.

• Assignee: Harsh J
• Reporter: Erik Steffl
• Created: 02/Jun/10 22:24
• Updated: 03/Mar/17 10:01
• Resolved: 03/Mar/17 09:42

Description

Following configuration keys in CommonConfigurationKeysPublic.java (former CommonConfigurationKeys.java):

public static final String IO_SORT_MB_KEY = "io.sort.mb";
public static final String IO_SORT_FACTOR_KEY = "io.sort.factor";

are partially moved:

• they were renamed to mapreduce.task.io.sort.mb and mapreduce.task.io.sort.factor respectively

• they were moved to mapreduce project, documented in mapred-default.xml

However:

• they are still listed in CommonConfigurationKeysPublic.java as quoted above

• strings "io.sort.mb" and "io.sort.factor" are used in SequenceFile.java in Hadoop Common project

Not sure what the solution is, these constants should probably be removed from CommonConfigurationKeysPublic.java but I am not sure what's the best solution for SequenceFile.java.

Issue Links

• incorporates: MAPREDUCE-2703 Add a doc note about changes to io.sort.mb and io.sort.factor config params for SequenceFile sorting in DistCp
• incorporates: MAPREDUCE-2622 Remove the last remaining reference to "io.sort.mb"
• links to: GitHub Pull Request #146