[YARN-3508] Prevent processing preemption events on the main RM dispatcher - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: 2.6.0
Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
Component/s: resourcemanager, scheduler
Labels:
None

Target Version/s:

2.7.2
Hadoop Flags:

Reviewed

Description

We recently saw the RM for a large cluster lag far behind on the AsyncDispacher event queue. The AsyncDispatcher thread was consistently blocked on the highly-contended CapacityScheduler lock trying to dispatch preemption-related events for RMContainerPreemptEventDispatcher. Preemption processing should occur on the scheduler event dispatcher thread or a separate thread to avoid delaying the processing of other events in the primary dispatcher queue.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

YARN-3508.002.patch
31/May/15 09:25
14 kB
Varun Saxena
YARN-3508.01.patch
28/May/15 22:55
13 kB
Varun Saxena
YARN-3508.03.patch
29/Jun/15 19:03
26 kB
Varun Saxena
YARN-3508.04.patch
29/Jun/15 21:25
26 kB
Varun Saxena
YARN-3508.05.patch
30/Jun/15 21:17
33 kB
Varun Saxena
YARN-3508.06.patch
30/Jun/15 23:39
33 kB
Varun Saxena
YARN-3508-branch-2.7.01.patch
02/Jul/15 09:40
29 kB
Varun Saxena

Activity

People

Assignee:: Varun Saxena

Reporter:: Jason Darrell Lowe

Votes:: 0 Vote for this issue

Watchers:: 18 Start watching this issue

Dates

Created:: 20/Apr/15 16:04

Updated:: 02/Dec/16 00:29

Resolved:: 02/Jul/15 18:09