[FLINK-9013] Document yarn.containers.vcores only being effective when adapting YARN config

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.6.1, 1.7.0
• Component/s: Deployment / YARN, Documentation
• Labels: pull-request-available
• Assignee: Dawid Wysakowicz
• Reporter: Nico Kruber
• Created: 16/Mar/18 15:17
• Updated: 16/Aug/18 07:21
• Resolved: 16/Aug/18 07:20

Description

Even after specifying yarn.containers.vcores and having Flink request such a container from YARN, it may not take these into account at all and return a container with 1 vcore.

The YARN configuration needs to be adapted to take the vcores into account, e.g. by setting the FairScheduler in yarn-site.xml:

<property>
  <name>yarn.resourcemanager.scheduler.class&lt;/name>
  <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler</value>
</property>

This fact should be documented at least at the configuration parameter documentation of yarn.containers.vcores.

Issue Links

• links to: GitHub Pull Request #6294
• links to: GitHub Pull Request #6294