Capacity Scheduler Queue Capacity

Table of Contents

1. Capacity Resource Vector

1.1. Motivation

CapacityResourceVector is a universal capacity resource, which is parsed for each queue from their capacity config. Currently, queues define capacity modes (absolute resource, percentage, weight), with different parsing locations and convoluted, ill-defined calculation methodology. There is no easy way to introduce a new capacity mode or the mixed mode. CapacityResourceVector will provide a way to centralize the effective resource calculation in a flexible and extendable way.

1.2. Design

The idea is to provide a capacity type for each resource. The materialized simplified interface of the class is:

public class QueueCapacityResourceVector {
    private Resource resource;
    private Map<String, QueueCapacityResourceType> queueCapacityResourceTypes;
}

1.3. Backward compatibility

It is a natural fit for all existing capacity configuration. For example:

  • root.example.capacity 50

    [memory=50%, vcores=50%, custom=50%]

  • root.example.capacity 6w

    [memory=6w, vcores=6w, custom=6w]

  • root.example.capacity [memory=12Gi, vcores=6, custom=10]

    Parsed as is, without any syntactic sugar

Author: Andras Gyori

Created: 2021-08-31 Tue 12:35