[YARN-10504] Implement weight mode in Capacity Scheduler

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: capacity scheduler
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: Benjamin Teke
• Reporter: Benjamin Teke
• Created: 30/Nov/20 16:44
• Updated: 12/Feb/24 01:02
• Resolved: 12/Jan/21 01:58

Description

To allow the possibility to flexibly create queues in Capacity Scheduler a weight mode should be introduced. The existing {{capacity }}property should be used with a different syntax, i.e:

root.users.capacity = (1.0) or ~1.0 or ^1.0 or @1.0

root.users.capacity = 1.0w

root.users.capacity = w:1.0

Weight support should not impact the existing functionality.

 

The new functionality should:

• accept and validate the new weight values
• enforce a singular mode on the whole queue tree
• (re)calculate the relative (percentage-based) capacities based on the weights during launch and every time the queue structure changes

Issue Links

• causes: YARN-10694 Fix spotbugs warning in CapacitySchedulerConfiguration.java