Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.20.205.0
-
None
-
None
Description
In the current implementation Capacity, MaxCapacity are the two main parameters that can be used for sharing resources on the cluster. But there is no feature to protect the resource of a queue. In the absence of pre-emption as a feature, if someone decides to reserve its capacity for its exclusive use, while rest of the queues are set for sharing, it is not simply possible.
If we choose one queue to be set for exclusive use by setting Capacity=MaxCapacity, this queue gets penalized, if others choose to set for sharing with their MaxCapacity more than capacity. This is an unintended consequence. On the other hand , if every queue is set to its Capacity=MaxCapacity the cluster is being partitioned into distinct virtual clusters.
So, the suggestion is, to add an additional queue parameter <queue_type> with value <exclusive> or <shared>. By default a queue is always shared type. If queue_type is exclusive, the capacity of this queue is non sharable and should be taken out from total capacity while scheduling jobs in other queues.