[YARN-7496] CS Intra-queue preemption user-limit calculations are not in line with LeafQueue user-limit calculations

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.8.2
• Fix Version/s: 2.8.3
• Hadoop Flags: Reviewed
• Assignee: Eric Payne
• Reporter: Eric Payne
• Created: 14/Nov/17 22:42
• Updated: 14/Oct/19 15:37
• Resolved: 24/Nov/17 04:21

Description

Only a problem in 2.8.

Preemption could oscillate due to the difference in how user limit is calculated between 2.8 and later releases.

Basically (ignoring ULF, MULP, and maybe others), the calculation for user limit on the Capacity Scheduler side in 2.8 is total used resources / number of active users while the calculation in later releases is total active resources / number of active users. When intra-queue preemption was backported to 2.8, it's calculations for user limit were more aligned with the latter algorithm, which is in 2.9 and later releases.

Issue Links

• is related to: YARN-7469 Capacity Scheduler Intra-queue preemption: User can starve if newest app is exactly at user limit