[YARN-2257] Add user to queue mappings to automatically place users' apps into specific queues

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: scheduler
• Labels: features
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Patrick Liu
• Created: 07/Jul/14 08:05
• Updated: 26/Feb/20 05:29
• Resolved: 27/Mar/19 12:06

Description

Currently, the fair-scheduler supports two modes, default queue or individual queue for each user.
Apparently, the default queue is not a good option, because the resources cannot be managed for each user or group.
However, individual queue for each user is not good enough. Especially when connecting yarn with hive. There will be increasing hive users in a corporate environment. If we create a queue for a user, the resource management will be hard to maintain.

I think the problem can be solved like this:
1. Define user->queue mapping in Fair-Scheduler.xml. Inside each queue, use aclSubmitApps to control user's ability.
2. Each time a user submit an app to yarn, if the user has mapped to a queue, the app will be scheduled to that queue; otherwise, the app will be submitted to default queue.
3. If the user cannot pass aclSubmitApps limits, the app will not be accepted.

Issue Links

• incorporates: YARN-2411 [Capacity Scheduler] support simple user and group mappings to queues
• Is contained by: YARN-8967 Change FairScheduler to use PlacementRule interface
• is related to: YARN-8792 Revisit FairScheduler QueuePlacementPolicy
• is superceded by: YARN-8794 QueuePlacementPolicy add more rules