[YARN-8945] Calculation of maximum applications should respect specified and global maximum applications for absolute resource

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.2.0
• Component/s: capacityscheduler
• Assignee: Tao Yang
• Reporter: Tao Yang
• Created: 26/Oct/18 02:17
• Updated: 09/Nov/18 15:56

Description

Currently maximum applications is expected to be calculated as follow according to priority when using percentage based capacity：
(1) equals specified maximum applications for queues
(2) equals global maximum applications
(3) calculated as queue-capacity * maximum-system-applications
But for absolute resource configuration, maximum applications is calculated as (3) in ParentQueue#deriveCapacityFromAbsoluteConfigurations, this is a strict limit for high max-capacity and low capacity queues which have little guaranteed resources but want to use lots of share resources. So I propose to share the maximum applications calculation of percentage based capacity, absolute resource can call the same calculation if necessary.

Issue Links

• is related to: YARN-5881 [Umbrella] Enable configuration of queue capacity in terms of absolute resources