[PIG-4626] [Pig on Tez] OOM in case of multiple outputs and POPartialAgg

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Fix Version/s: 0.18.0
• Assignee: Rohini Palaniswamy
• Reporter: Rohini Palaniswamy
• Created: 07/Jul/15 23:31
• Updated: 24/May/17 23:03

Description

tez.task.scale.memory.reserve-fraction is 0.3 by default which assumes 70% of memory is available for io.sort.mb. If map is configured as 1G Xmx and io.sort.mb as 256MB, and there is group by on 3 different keys it has 3 different outputs and Tez's WeightedScalingMemoryDistributor allocates 256MB to each of the outputs which leaves very less memory for Pig and POPartialAgg (which requires 20% of memory) and sometime leads to OOM. This also causes SpillableMemoryManager to be invoked often as its threshold is set to 70% of memory.

Need to set tez.task.scale.memory.reserve-fraction as 0.5 for all pig jobs and 0.6 in case there is POPartialAgg in combiner plan.

Issue Links

• requires: TEZ-2710 TEZ_TASK_SCALE_MEMORY_RESERVE_FRACTION should be configurable at vertex level
• requires: TEZ-3286 Allow clients to set processor reserved memory per vertex (instead of per container)

Sub-Tasks

• Reduce tez memory.reserve-fraction and clear spillables for better memory utilization: Closed