[YARN-2380] The normalizeRequests method in SchedulerUtils always resets the vCore to 1

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.4.0
• Component/s: resourcemanager
• Assignee: Jian Fang
• Reporter: Jian Fang
• Created: 01/Aug/14 21:09
• Updated: 08/May/15 21:15

Description

I added some log info to the method normalizeRequest() as follows.

public static void normalizeRequest(
ResourceRequest ask,
ResourceCalculator resourceCalculator,
Resource clusterResource,
Resource minimumResource,
Resource maximumResource,
Resource incrementResource)

{ LOG.info("Before request normalization, the ask capacity: " + ask.getCapability()); Resource normalized = Resources.normalize( resourceCalculator, ask.getCapability(), minimumResource, maximumResource, incrementResource); LOG.info("After request normalization, the ask capacity: " + normalized); ask.setCapability(normalized); }

The resulted log showed that the vcore in ask was changed from 2 to 1.

2014-08-01 20:54:15,537 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC Server handler 4 on 9024): Before request normalization, the ask capacity: <memory:1536, vCores:2>
2014-08-01 20:54:15,537 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerUtils (IPC Server handler 4 on 9024): After request normalization, the ask capacity: <memory:1536, vCores:1>

The root cause is the DefaultResourceCalculator calls Resources.createResource(normalizedMemory) to regenerate a new resource with vcore = 1.

This bug is critical and it leads to the mismatch of the request resource and the container resource and many other potential issues if the user requests containers with vcore > 1.

Issue Links

• blocks: REEF-35 Include vCore-checking in the YARN Container matching algorithm
• blocks: TEZ-704 Tez MapReduce job hangs if setting mapreduce.reduce.cpu.vcores to be 2