Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4866

ShuffleRamManager is limited to 2Gb of memory - we should increase that

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: mrv1
    • Labels:
    • Environment:

      linux, 64bits cpu, more than 2Gb of memory for each reducer tasks

      Description

      Inside the org.apache.hadoop.mapred.ReduceTask.java, the ShuffleRamManager is limited to allocate up to 2Gb of memory during the shuffle phase.
      We should be able to allocate more, to take advantage of the full memory we have on servers.

        Attachments

        1. M4866-0.patch
          1 kB
          Chris Douglas
        2. MAPREDUCE-4866-INCOMPLETE.patch
          4 kB
          Varene Olivier

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              ov Varene Olivier
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: