[YARN-3508] Prevent processing preemption events on the main RM dispatcher

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
• Component/s: resourcemanager, scheduler
• Target Version/s: 2.7.2
• Hadoop Flags: Reviewed
• Assignee: Varun Saxena
• Reporter: Jason Lowe
• Created: 20/Apr/15 16:04
• Updated: 02/Dec/16 00:29
• Resolved: 02/Jul/15 18:09

Description

We recently saw the RM for a large cluster lag far behind on the AsyncDispacher event queue. The AsyncDispatcher thread was consistently blocked on the highly-contended CapacityScheduler lock trying to dispatch preemption-related events for RMContainerPreemptEventDispatcher. Preemption processing should occur on the scheduler event dispatcher thread or a separate thread to avoid delaying the processing of other events in the primary dispatcher queue.