[YARN-5892] Support user-specific minimum user limit percentage in Capacity Scheduler

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 2.8.2, 3.0.0-alpha3
• Component/s: capacityscheduler
• Hadoop Flags: Reviewed
• Assignee: Eric Payne
• Reporter: Eric Payne
• Created: 16/Nov/16 20:32
• Updated: 25/Oct/19 20:27
• Resolved: 28/Jul/17 14:41

Description

Currently, in the capacity scheduler, the minimum-user-limit-percent property is per queue. A cluster admin should be able to set the minimum user limit percent on a per-user basis within the queue.

This functionality is needed so that when intra-queue preemption is enabled (YARN-4945 / YARN-2113), some users can be deemed as more important than other users, and resources from VIP users won't be as likely to be preempted.

For example, if the getstuffdone queue has a MULP of 25 percent, but user jane is a power user of queue getstuffdone and needs to be guaranteed 75 percent, the properties for getstuffdone and jane would look like this:

  <property>
    <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
    <value>25</value>
  </property>

  <property>
    <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
    <value>75</value>
  </property>

Issue Links

• relates to: YARN-5889 Improve and refactor user-limit calculation in capacity scheduler