[MAPREDUCE-5906] Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent"

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.0
• Fix Version/s: 2.6.0
• Labels: newbie
• Target Version/s: 2.6.0
• Assignee: Akira Ajisaka
• Reporter: Akira Ajisaka
• Created: 27/May/14 08:50
• Updated: 01/Dec/14 03:11
• Resolved: 14/Aug/14 18:08

Description

In MergeManagerImpl.java, the default value of MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT (=mapreduce.reduce.shuffle.input.buffer.percent) looks 0.90.

  final float maxInMemCopyUse =
    jobConf.getFloat(MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT, 0.90f);

However, the actual default value is 0.70 in mapred-default.xml.

  <name>mapreduce.reduce.shuffle.input.buffer.percent</name>
  <value>0.70</value>