[PIG-4948] Pig on Tez AM use too much memory on a small cluster

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.17.0, 0.16.1
• Component/s: tez
• Hadoop Flags: Reviewed
• Assignee: Daniel Dai
• Reporter: Daniel Dai
• Created: 14/Jul/16 22:31
• Updated: 21/Jun/17 09:15
• Resolved: 19/Jul/16 01:29

Description

Pig will use 1024M for AM memory even if "tez.am.resource.memory.mb" is smaller. On a small cluster such as sandbox, the total memory is very small (eg, 2g) and this will make the capacity full and not able to run any job.

Issue Links

• is related to: PIG-4962 Estimate smaller Tez AM memory for smaller count of tasks