-1 overall

This message was automatically generated.

-1 overall

This message was automatically generated.

Checkstyle is showing incorrect result it seems. There were 53 checkstyle issues, both, before and after the patch.

-1 overall

This message was automatically generated.

Trying to better understand this problem: I'm not sure where is bottleneck. If CapacityScheduler becomes bottleneck, move preemption events out of main RM dispatcher doesn't help. This approach only helps when main dispatcher is bottleneck.

And a parallel thing we can do is to optimize number of preemption event. Currently, if a container sits in to-preempt list, before it is get preempted, one event will be sent to scheduler for every few seconds, we can reduce frequency of this event.

Yes, it's not a cure-all to move the preemption processing to the scheduler event queue when the scheduler is the bottleneck, but we do have separate event queues for a reason. If it didn't matter who was the bottleneck then we'd just have one event queue for everything, correct? The scheduler event queue is primarily blocked by the big scheduler lock, and IMHO we should dispatch events that need that lock to that queue. Doing otherwise starts to couple the two event dispatchers together and we might as well just have the one event queue to rule them all.

I think this make sense to move preemption events from main dispatcher to the scheduler dispatcher. Otherwise, any non-scheduler events on main dispatcher will be waiting for the preemption events to grab the scheduler lock, which is unnecessary.

Just to clarify, in the implementation I have spawned a new preemption dispatcher thread instead of posting preemption events to scheduler dispatcher.
This is because IMHO container preemption events should have priority over scheduler events. This approach though would make this one extra thread contending for scheduler lock.

Another approach though would be to post events preemption events to scheduler dispatcher. And have a LinkedBlockingDeque for storing events instead. This way preemption events can be posted to front of queue. However, linked blocking deque uses a single lock for put and take operations whereas linked blocking queue uses 2 different locks for these 2 operations making the latter better from a performance viewpoint.

Jason Lowe, Jian He, Wangda Tan, thoughts on the approaches mentioned above ?

The problem I see in the clubbing with scheduler events is if there is many scheduler events already in the event queue then it delays pre-emption events to trigger. As Varun Saxena said, container preemption events should be considered as higher priority than scheduler events. Having separate event disaptcher for preemptiong events would allow preemption events to participate in obtaining the lock in-earlier-stages rather then waiting for scheuduler events queue to complete. I think current patch approach make sense to me i.e having individual dispatcher thread for preemption events.

Jason Lowe/Jian He/Wangda Tan/Rohith Sharma K S, can one of the commiters have a look at this ?

I tent to support Jason Lowe and Jian He's suggestion, make preemption events directly goes to scheduler event queue.

I think we cannot assume preemption events have higher priority than other events, in most cases, preemption events are just notify AM about something will happen. And manage two queues for scheduler can be complex, how to balance them, etc. To reduce complex, I suggest to only maintain one queue for scheduler until we have to.

Wangda Tan, ok if thats the consensus, I will do so.

Moving out all non-critical / non-blocker issues that didn't make it out of 2.7.1 into 2.7.2.

-1 overall

This message was automatically generated.

Varun Saxena,
It seems to me this patch doesn't work as what I expected:

I think we PreemptionPolicy should directly send events to scheduler event dispatcher, so events won't go to rmDispatcher. What Jason Lowe/Jian He suggested should be the same.

Now my understanding of this patch is, preemption events still go to rmDispatcher, and rmDispatcher sends events to schedulerDispatcher:

  @SuppressWarnings("unchecked")
  public void serviceInit(Configuration conf) throws Exception {
    scheduleEditPolicy.init(conf, rmContext.getDispatcher().getEventHandler(),
        (PreemptableResourceScheduler) rmContext.getScheduler());
    this.monitorInterval = scheduleEditPolicy.getMonitoringInterval();
    super.serviceInit(conf);
  }

I think one solution is exposing scheduler events dispatcher to RMContext interface, so schedulerEditPolicy can use the schedulerDispatcher do initialization.

And could you take a look at failed tests, it looks like related to the patch.

Wangda Tan, the test failures seem unrelated. Had submitted patch again. Lets see what that gives.

Now my understanding of this patch is, preemption events still go to rmDispatcher, and rmDispatcher sends events to schedulerDispatcher.

Yes, you are correct.

The issue in the JIRA is that on picking up the ContainerPreempt Event from RM Dispatcher Queue, this event is processed by the RM Dispatcher thread itself(which can block this dispatcher) instead of putting it in another dispatcher queue. This was blocking the RM Dispatcher thread because it was waiting for scheduler lock. Hence other events in the queue were getting delayed.

The current design is such that different dispatchers(based on event type) are registered to main RM Dispatcher. And RM Dispatcher then dispatches these events to appropriate registered dispatchers. Although, its an additional step(and may cost some extra processing time), I think this has been done to have a single point of contact for dispatching events across RM.

We can open an interface for posting events directly to Scheduler Events as well. That ofcourse would be faster. But then this change should be made for all the scheduler events not only container preemption events. Thoughts ?

I mean,
We can open an interface for posting events directly to Scheduler Dispatcher as well.

-1 overall

This message was automatically generated.

Thanks for explanation, Varun Saxena.

Currently, YARN's non-preemption scheduler events are sent to central RM dispatcher and then pushed to scheduler event queue. RM dispatcher won't wait for these events to be handled, so the performance should be good.

Preemption event is different, RMContainerPreemptEventDispatcher is a synchronized dispatcher, when RMDispatcher receive these events, it will wait for these events proceeded.

My suggestion is to make ContainerPreemptEventType as a part of SchedulerEventType, and ProportionalCapacityPreemptionPolicy sends the event to RMDispatcher. So these events will be sent to SchedulerEventQueue, and not block central RMDispatcher.

Thoughts?

Wangda Tan, yes this is what the patch does and addresses the issue in hand. I thought you want me to put events directly to Scheduler Event Queue and hence bypass Central RM Dispatcher Queue.

Anyways just to explain, below is what I have done.
I have removed ContainerPreemptEventType and moved all these events to SchedulerEventType.
I have also removed RMContainerPreemptEventDispatcher because that processes the events synchronously as part of RMDispatcher Event Thread.

As per patch the events from ProportionalCapacityPreemptionPolicy are sent to central RM Dispatcher and as the event type is SchedulerEventType, the preemption events are sent to scheduler event queue.
Then the Scheduler Dispatcher picks up these events from scheduler event queue and call CapacityScheduler#handle which then calls the relevant method to handle different preemption events.

I added a test case too for this behavior by adding a new test class TestRMDispatcher.

I think my comment confusing you: https://issues.apache.org/jira/browse/YARN-3508?focusedCommentId=14606576&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14606576. I was thinking yesterday that we need to directly send preemption events to scheduler event dispatcher. But if we expose scheduler event dispatcher in RMContext, more event dispatcher will be exposed in the future, that is against the reason why we have a central RM dispatcher.

You patch basically looks good, some minor comments:

• Instead of passing EventHandler<SchedulerEvent> to Proportional.., you should pass RMContext.
• Not caused by your patch, save e.getKey() to ApplicationAttemptId in containerBasedPreemptOrKill avoid multiple getKey
• Source: ProportionalCapacityPreemptionPolicy, should be SchedulingEditPolicy

Other looks good to me.

Instead of passing EventHandler<SchedulerEvent> to Proportional.., you should pass RMContext.

Ok, IIUC you want me to pass RMContext to ProportionalCapacityPreemptionPolicy and get event handler from there. Correct ?

Any reason you want this changed ? Even if we pass RMContext we will have to eventually get EventHandler from that because SchedulingEditPolicy#init wants that. And anyways ProportionalCapacityPreemptionPolicy constructors are used only from tests.

Ok, IIUC you want me to pass RMContext to ProportionalCapacityPreemptionPolicy and get event handler from there. Correct ?

Yes

Any reason you want this changed?

RM components have a reference to RMContext is a convention, you can see most of RM components have RMContext. And this can avoid some future code changes if Proportional..Policy needs more information beyond dispatcher.

And anyways ProportionalCapacityPreemptionPolicy constructors are used only from tests.

Init is used by production code, I suggest to make constructor/init signature consistent.

Wangda Tan,

Init is used by production code, I suggest to make constructor/init signature consistent.

Yes SchedulingEditPolicy#init and constructor are consistent. init takes EventHandler as argument just like the constructor. Anyways I will make the change.
I will therefore, change SchedulingEditPolicy#init as well (to make it consistent with the constructor).

By the way, this is how SchdulingEditPolicy#init looked before my change.

  public void init(Configuration config, EventHandler<ContainerPreemptEvent> dispatcher, PreemptableResourceScheduler scheduler);

And this is how ProportionalCapacityPreemptionPolicy constructor looked.

   public ProportionalCapacityPreemptionPolicy(Configuration config, EventHandler<ContainerPreemptEvent> dispatcher, CapacityScheduler scheduler)

So I merely changed EventHandler<ContainerPreemptEvent> to EventHandler<SchedulerEvent> in the function signature.

Will make the change and update patch shortly.

Wangda Tan, updated the patch as per your comments. Kindly review.

BTW, as we are passing RMContext into SchedulingEditPolicy#init, we can do away with PreemptableResourceScheduler as its parameter as well because we can get scheduler instance from RMContext as well.

-1 overall

This message was automatically generated.

Fixed one checkstyle issue

-1 overall

This message was automatically generated.

Wangda Tan, the checkstyle issue is because of the formatting of following piece of code.

 switch(event.getType()) {
    case NODE_ADDED:
    {
      NodeAddedSchedulerEvent nodeAddedEvent = (NodeAddedSchedulerEvent)event;
      addNode(nodeAddedEvent.getAddedRMNode());
      recoverContainersOnNode(nodeAddedEvent.getContainerReports(),
        nodeAddedEvent.getAddedRMNode());
    }
    break;
    case NODE_REMOVED:
    {
      NodeRemovedSchedulerEvent nodeRemovedEvent = (NodeRemovedSchedulerEvent)event;
      removeNode(nodeRemovedEvent.getRemovedRMNode());
    }
    break;

I added the code in the same format as previous piece of code. If I fix this issue I will have to change code unrelated to my change as well.
Do you want me to fix it ? Or ignore this checkstyle issue ?

Wangda Tan, the timed out test TestNodeLabelContainerAllocation is unrelated and will be handled by YARN-3848.
Maybe you can review that JIRA too

Thanks for update, Varun Saxena.

Checkstyle is fine to me, the patch generally looks good. Jian He/Jason Lowe, could you take a look also?

lgtm, thanks Varun Saxena !

Committing..

Varun Saxena,

I just pushed this change to trunk/branch-2. Could you rebase the patch to branch-2.7 and run tests of RM? I will resolve the patch after it went to branch-2.7.

Thanks,
Wangda

FAILURE: Integrated in Hadoop-trunk-Commit #8109 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8109/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java

Thanks for the review and commit Wangda Tan. Will make the changes for 2.7 and update

Thanks for the review Jian He

Wangda Tan, updated patch for branch-2.7

Committed to trunk/branch-2/branch-2.7, thanks Varun Saxena and review from Rohith Sharma K S, Jian He, Jason Lowe.

FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #246 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/246/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java

SUCCESS: Integrated in Hadoop-Yarn-trunk #976 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/976/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java

FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #244 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/244/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #234 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/234/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java

SUCCESS: Integrated in Hadoop-Hdfs-trunk #2173 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2173/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/CHANGES.txt

FAILURE: Integrated in Hadoop-Mapreduce-trunk #2192 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2192/)
YARN-3508. Prevent processing preemption events on the main RM dispatcher. (Varun Saxena via wangda) (wangda: rev 0e4b06690ff51fbde3ab26f68fde8aeb32af69af)

• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/CHANGES.txt
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingMonitor.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMDispatcher.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/event/SchedulerEventType.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/ProportionalCapacityPreemptionPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/SchedulingEditPolicy.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/ContainerPreemptEvent.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/monitor/capacity/TestProportionalCapacityPreemptionPolicyForNodePartitions.java
• hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java