[MAPREDUCE-4866] ShuffleRamManager is limited to 2Gb of memory - we should increase that

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.20.2
• Component/s: mrv1
• Labels: patch

Environment

linux, 64bits cpu, more than 2Gb of memory for each reducer tasks

• Assignee: Unassigned
• Reporter: Varene Olivier
• Created: 10/Dec/12 10:01
• Updated: 05/Jan/14 03:12

Description

Inside the org.apache.hadoop.mapred.ReduceTask.java, the ShuffleRamManager is limited to allocate up to 2Gb of memory during the shuffle phase.
We should be able to allocate more, to take advantage of the full memory we have on servers.