Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-45 [Preemption] Scheduler feedback to AM to release containers
  3. YARN-1184

ClassCastException is thrown during preemption When a huge job is submitted to a queue B whose resources is used by a job in queueA

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      preemption is enabled.
      Queue = a,b
      a capacity = 30%
      b capacity = 70%

      Step 1: Assign a big job to queue a ( so that job_a will utilize some resources from queue b)
      Step 2: Assigne a big job to queue b.

      Following exception is thrown at Resource Manager

      2013-09-12 10:42:32,535 ERROR [SchedulingMonitor (ProportionalCapacityPreemptionPolicy)] yarn.YarnUncaughtExceptionHandler (YarnUncaughtExceptionHandler.java:uncaughtException(68)) - Thread Thread[SchedulingMonitor (ProportionalCapacityPreemptionPolicy),5,main] threw an Exception.
      java.lang.ClassCastException: java.util.Collections$UnmodifiableSet cannot be cast to java.util.NavigableSet
      	at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.getContainersToPreempt(ProportionalCapacityPreemptionPolicy.java:403)
      	at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.containerBasedPreemptOrKill(ProportionalCapacityPreemptionPolicy.java:202)
      	at org.apache.hadoop.yarn.server.resourcemanager.monitor.capacity.ProportionalCapacityPreemptionPolicy.editSchedule(ProportionalCapacityPreemptionPolicy.java:173)
      	at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor.invokePolicy(SchedulingMonitor.java:72)
      	at org.apache.hadoop.yarn.server.resourcemanager.monitor.SchedulingMonitor$PreemptionChecker.run(SchedulingMonitor.java:82)
      	at java.lang.Thread.run(Thread.java:662)
      
      

        Attachments

        1. Y1184-0.patch
          1 kB
          Christopher Douglas
        2. Y1184-1.patch
          1 kB
          Christopher Douglas

          Activity

            People

            • Assignee:
              cdouglas Christopher Douglas
              Reporter:
              andreina J.Andreina
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: