[YARN-8994] Fix race condition between move app and queue cleanup in Fair Scheduler

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.3.0
• Component/s: fairscheduler
• Hadoop Flags: Reviewed
• Assignee: Wilfred Spiegelenburg
• Reporter: Wilfred Spiegelenburg
• Created: 09/Nov/18 02:31
• Updated: 26/Feb/20 05:29
• Resolved: 05/Dec/18 22:44

Description

Similar to YARN-8990 and also introduced by YARN-8191 there is a race condition while moving an application. The pre-move check looks for the queue and when it finds the queue it progresses. The real move then retrieves the queue and does further check before updating the app and queues.

The move uses the retrieved queue object but the queue could have become empty while checks are performed. If the cleanup runs at that same time the app will be moved to a deleted queue and lost.

Issue Links

• is caused by: YARN-8191 Fair scheduler: queue deletion without RM restart
• relates to: YARN-8990 Fix fair scheduler race condition in app submit and queue cleanup