[YARN-9936] Support vector of capacity percentages in Capacity Scheduler configuration

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: capacity scheduler
• Target Version/s: 3.3.0, 3.2.2
• Assignee: Zoltan Siegl
• Reporter: Zoltan Siegl
• Created: 28/Oct/19 09:27
• Updated: 27/Nov/19 07:47

Description

Currently, the Capacity Scheduler queue configuration supports two ways to set queue capacity.

• In percentage of all available resources as a float ( eg. 25.0 ) means 25% of the resources of its parent queue for all resource types equally (eg. 25% of all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster) The percentages of all queues has to add up to 100%.
• In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4 ). The amount of all resources in the queues has to be less than or equal to all resources in the cluster.

Apart from these two already existing ways, there is a demand to add capacity percentage of each available resource type separately. (eg. memory=20%,vcores=40%,yarn.io/gpu=100%).
At the same time, a similar concept should be included with queues maximum-capacity as well.

Issue Links

• is duplicated by: YARN-9891 Capacity scheduler: enhance capacity / maximum-capacity setting