[YARN-1184] ClassCastException is thrown during preemption When a huge job is submitted to a queue B whose resources is used by a job in queueA

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.1.0-beta
• Fix Version/s: 2.1.1-beta
• Component/s: capacityscheduler, resourcemanager
• Hadoop Flags: Reviewed
• Assignee: Christopher Douglas
• Reporter: J.Andreina
• Created: 12/Sep/13 11:06
• Updated: 13/May/14 21:23
• Resolved: 17/Sep/13 01:33

Description

preemption is enabled.
Queue = a,b
a capacity = 30%
b capacity = 70%

Step 1: Assign a big job to queue a ( so that job_a will utilize some resources from queue b)
Step 2: Assigne a big job to queue b.

Following exception is thrown at Resource Manager

2013-09-12 10:42:32,535 ERROR [SchedulingMonitor (ProportionalCapacityPreemptionPolicy)] yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(68)) - Thread Thread[SchedulingMonitor (ProportionalCapacityPreemptionPolicy),5,main] threw an Exception.
java.lang.ClassCastException: java.util.Collections$UnmodifiableSet cannot be cast to java.util.NavigableSet
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.getContainersToPreempt(ProportionalCapacityPreemptionPolicy.java:403)
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:202)
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:173)
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:72)
	at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PreemptionChecker.run(SchedulingMonitor.java:82)
	at java.lang.Thread.run(Thread.java:662)