Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5731

Preemption calculation is not accurate when reserved containers are present in queue.

    Details

    • Hadoop Flags:
      Reviewed

      Description

      YARN Capacity Scheduler does not kick Preemption under below scenario.
      Two queues A and B each with 50% capacity and 100% maximum capacity and user limit factor 2. Minimum Container size is 1536MB and total cluster resource is 40GB. Now submit the first job which needs 1536MB for AM and 9 task containers each 4.5GB to queue A. Job will get 8 containers total (AM 1536MB + 7 * 4.5GB = 33GB) and the cluster usage is 93.8% and the job has reserved a container of 4.5GB.
      Now when next job (1536MB for AM and 9 task containers each 4.5GB) is submitted onto queue B. The job hangs in ACCEPTED state forever and RM scheduler never kicks in Preemption. (RM UI Image 2 attached)
      Test Case:

      ./spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client --queue A --executor-memory 4G --executor-cores 4 --num-executors 9 ../lib/spark-examples*.jar 1000000

      After a minute..

      ./spark-submit --class org.apache.spark.examples.SparkPi --master yarn-client --queue B --executor-memory 4G --executor-cores 4 --num-executors 9 ../lib/spark-examples*.jar 1000000

      Credit to: Prabhu Joseph for bug investigation and troubleshooting.

        Attachments

        1. YARN-5731.001.patch
          12 kB
          Wangda Tan
        2. YARN-5731.002.patch
          10 kB
          Wangda Tan
        3. YARN-5731.addendum.003.patch
          6 kB
          Wangda Tan
        4. YARN-5731.addendum.004.patch
          7 kB
          Wangda Tan
        5. YARN-5731.branch-2.002.patch
          10 kB
          Wangda Tan
        6. YARN-5731.branch-2.8.004.patch
          12 kB
          Wangda Tan
        7. YARN-5731-branch-2.8.001.patch
          34 kB
          Sunil Govindan
        8. YARN-5731-branch-2.8.001.patch
          6 kB
          Wangda Tan

          Activity

            People

            • Assignee:
              leftnoteasy Wangda Tan
              Reporter:
              sunilg Sunil Govindan
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: