[YARN-4606] CapacityScheduler: applications could get starved because computation of #activeUsers considers pending apps

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.8.0, 2.7.1
• Fix Version/s: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2
• Component/s: capacity scheduler, capacityscheduler
• Assignee: Manikandan R
• Reporter: Karam Singh
• Created: 19/Jan/16 09:43
• Updated: 14/Oct/19 15:38
• Resolved: 25/Jul/18 19:13

Description

Currently, if all applications belong to same user in LeafQueue are pending (caused by max-am-percent, etc.), ActiveUsersManager still considers the user is an active user. This could lead to starvation of active applications, for example:

• App1(belongs to user1)/app2(belongs to user2) are active, app3(belongs to user3)/app4(belongs to user4) are pending
• ActiveUsersManager returns #active-users=4
• However, there're only two users (user1/user2) are able to allocate new resources. So computed user-limit-resource could be lower than expected.

Issue Links

• relates to: YARN-8660 Fair/FIFOSchedulers: applications could get starved because computation of #activeUsers considers pending apps