Currently, the Capacity Scheduler queue configuration supports two ways to set queue capacity.
- In percentage of all available resources as a float ( eg. 25.0 ) means 25% of the resources of its parent queue for all resource types equally (eg. 25% of all memory, 25% of all CPU cores, and 25% of all available GPU in the cluster) The percentages of all queues has to add up to 100%.
- In an absolute amount of resources ( e.g. memory=4GB,vcores=20,yarn.io/gpu=4 ). The amount of all resources in the queues has to be less than or equal to all resources in the cluster.Actually, the above is not supported, we only support memory and vcores now in absolute mode, we should extend in
Apart from these two already existing ways, there is a demand to add capacity percentage of each available resource type separately. (eg. memory=20%,vcores=40%,yarn.io/gpu=100%).
At the same time, a similar concept should be included with queues maximum-capacity as well.