Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-4866

ShuffleRamManager is limited to 2Gb of memory - we should increase that

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 0.20.2
    • Fix Version/s: None
    • Component/s: mrv1
    • Labels:
    • Environment:

      linux, 64bits cpu, more than 2Gb of memory for each reducer tasks

      Description

      Inside the org.apache.hadoop.mapred.ReduceTask.java, the ShuffleRamManager is limited to allocate up to 2Gb of memory during the shuffle phase.
      We should be able to allocate more, to take advantage of the full memory we have on servers.

      1. MAPREDUCE-4866-INCOMPLETE.patch
        4 kB
        Varene Olivier
      2. M4866-0.patch
        1 kB
        Chris Douglas

        Activity

          People

          • Assignee:
            Unassigned
            Reporter:
            Varene Olivier
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:

              Development