[YARN-10531] Be able to disable user limit factor for CapacityScheduler Leaf Queue

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: capacity scheduler
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Qi Zhu
• Reporter: Wangda Tan
• Created: 11/Dec/20 20:19
• Updated: 12/Feb/24 01:10
• Resolved: 26/Jan/21 16:37

Description

User limit factor is used to define max cap of how much resource can be consumed by single user.

Under Auto Queue Creation context, it doesn't make much sense to set user limit factor, because initially every queue will set weight to 1.0, we want user can consume more resource if possible. It is hard to pre-determine how to set up user limit factor. So it makes more sense to add a new value (like -1) to indicate we will disable user limit factor

Logic need to be changed is below:

(Inside LeafQueue.java)

    Resource maxUserLimit = Resources.none();
    if (schedulingMode == SchedulingMode.RESPECT_PARTITION_EXCLUSIVITY) {
      maxUserLimit = Resources.multiplyAndRoundDown(queueCapacity,
          getUserLimitFactor());
    } else if (schedulingMode == SchedulingMode.IGNORE_PARTITION_EXCLUSIVITY) {
      maxUserLimit = partitionResource;
    }