Usage of Capacity Scheduler Queue Priority

Configurations

Property Description
yarn.scheduler.capacity.<queue-path>.ordering-policy The ordering-policy of the parent queue. Can be one of utilization or priority-utilization. (see notes)
yarn.scheduler.capacity.<queue-path>.priority The priority of queue, higher means more important. It should be a valid integer value, 0 is default value
yarn.scheduler.capacity.<queue-path>.priority The priority of queue, higher means more important. It should be a valid integer value, 0 is default value
Property Description
yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled Do we allow under-utilized queue with higher priority to preempt queue with lower priority even if queue with lower priority is not satisfied. For example, two queues, a and b. A’s priority = 1, (a.used-capacity - a.reserved-capacity) = 40%. B’s priority = 0, b.used-capacity = 30%. Set this configuration to true to allow queue-a to preempt container from queue-b. (The reason why deduct reserved-capacity from used-capacity for queue with higher priority is: the reserved-capacity is just scheduler’s internal implementation to allocate large containers, it is not possible for application to use such reserved-capacity. It is possible that a queue with large container requests have a large number of containers but cannot allocate from any of them. But scheduler will make sure a satisfied queue will not preempt resource from any other queues. A queue is considered to be satisfied when queue’s used-capacity - reserved-capacity >= guaranteed-capacity.)
yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.reserved-container-delay-ms When a reserved container of an underutilized queue is created. Preemption will kick in after specified delay (in ms). The total time to preempt resources for a reserved container from higher priority queue will be: reserved-container-delay-ms + PREEMPTION_WAIT_TIME_BEFORE_KILL. This parameter is added to make preemption from lower priority queue which is underutilized to be more careful. This parameter takes effect when underutilized-preemption.enabled set to true.
yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.reserved-container-delay-ms When doing preemption from under-satisfied queues for priority queue. Do we allow move reserved container from one host to another? (Experimental config, by default it is false)

Notes

1) Queue priority-utilization ordering policy

For two queues under same parent with the same priority:

For two queues under same parent with different priorities:

2) Queue utilization ordering policy

For any of two queues under the same parent, the queue with the lowest used-capacity percentage gets the resources.

Example of configuration

<configuration>
   <property>
      <name>yarn.scheduler.capacity.root.ordering-policy</name>
      <value>priority-utilization</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.enabled</name>
      <value>true</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.allow-move-reservation</name>
      <value>false</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.ordering-policy.priority-utilization.underutilized-preemption.reserved-container-delay-ms</name>
      <value>5000</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.queues</name>
      <value>a,b,c,d</value>
      <description>The queues at the this level (root is the root queue).</description>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.a.capacity</name>
      <value>5</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.a.priority</name>
      <value>2</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.a.user-limit-factor</name>
      <value>10</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.a.maximum-capacity</name>
      <value>100</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.b.capacity</name>
      <value>10</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.b.priority</name>
      <value>1</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.b.maximum-capacity</name>
      <value>100</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.a.user-limit-factor</name>
      <value>5</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.c.capacity</name>
      <value>25</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.c.priority</name>
      <value>1</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.c.maximum-capacity</name>
      <value>100</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.c.user-limit-factor</name>
      <value>3</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.d.capacity</name>
      <value>60</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.d.maximum-capacity</name>
      <value>100</value>
   </property>
   <property>
      <name>yarn.scheduler.capacity.root.d.user-limit-factor</name>
      <value>2</value>
   </property>
</configuration>