[YARN-9936] Support vector of capacity percentages in Capacity Scheduler configuration

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: capacity scheduler
• Target Version/s: 3.4.0
• Assignee: Zoltan Siegl
• Reporter: Zoltan Siegl
• Created: 28/Oct/19 09:27
• Updated: 03/Dec/20 09:28

Description

Currently, the Capacity Scheduler queue configuration supports two ways to set queue capacity.

• In percentage of all available resources as a float ( eg. 25.0 ) means 25% of the resources of its parent queue for all resource types equally (eg. 25% of all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster) The percentages of all queues has to add up to 100%.
• In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4 ). The amount of all resources in the queues has to be less than or equal to all resources in the cluster.

Apart from these two already existing ways, there is a demand to add capacity percentage of each available resource type separately. (eg. memory=20%,vcores=40%,yarn.io/gpu=100%).
At the same time, a similar concept should be included with queues maximum-capacity as well.

Issue Links

• is duplicated by: YARN-9891 Capacity scheduler: enhance capacity / maximum-capacity setting
• is related to: YARN-10505 Extend the maximum-capacity property to react to weight mode changes