[TEZ-704] Tez MapReduce job hangs if setting mapreduce.reduce.cpu.vcores to be 2

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Zhenxiao Luo
• Created: 06/Jan/14 06:58
• Updated: 14/Apr/16 23:29

Description

When running Hive on Tez in EMR hadoop districution, which set mapreduce.reduce.cpu.vcores to be 2 in mapred-site.xml, all MapReduce jobs get hang, without any progress.

The problem seems like,
Hive On Tez sets mapreduce.reduce.cpu.vcores to 2, so its application
manager asks for a 2 cpu container, but the resource manager could not
allocate a node for it, since all it has are 1 cpu containers.

I saw the same problem when running MapReduce Job on Tez when setting mapreduce.reduce.cpu.vcores to be 2.

This could be a bug in Tez MapReduce code.

Issue Links

• is blocked by: YARN-2380 The normalizeRequests method in SchedulerUtils always resets the vCore to 1