Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-5906

Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent"

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.6.0
    • Component/s: None
    • Labels:
    • Target Version/s:

      Description

      In MergeManagerImpl.java, the default value of MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT (=mapreduce.reduce.shuffle.input.buffer.percent) looks 0.90.

        final float maxInMemCopyUse =
          jobConf.getFloat(MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT, 0.90f);
      

      However, the actual default value is 0.70 in mapred-default.xml.

        <name>mapreduce.reduce.shuffle.input.buffer.percent</name>
        <value>0.70</value>
      

        Attachments

        1. MAPREDUCE-5906.patch
          2 kB
          Akira Ajisaka

          Activity

            People

            • Assignee:
              aajisaka Akira Ajisaka
              Reporter:
              aajisaka Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: