Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-4945 [Umbrella] Capacity Scheduler Preemption Within a queue
  3. YARN-4781

Support intra-queue preemption for fairness ordering policy.

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.10.0, 2.9.2, 3.0.3, 2.8.5
    • scheduler
    • None
    • Reviewed

    Description

      We introduced fairness queue policy since YARN-3319, which will let large applications make progresses and not starve small applications. However, if a large application takes the queue’s resources, and containers of the large app has long lifespan, small applications could still wait for resources for long time and SLAs cannot be guaranteed.

      Instead of wait for application release resources on their own, we need to preempt resources of queue with fairness policy enabled.

      Attachments

        1. YARN-4781.001.patch
          21 kB
          Eric Payne
        2. YARN-4781.002.patch
          21 kB
          Eric Payne
        3. YARN-4781.003.patch
          23 kB
          Eric Payne
        4. YARN-4781.004.patch
          26 kB
          Eric Payne
        5. YARN-4781.005.patch
          26 kB
          Eric Payne
        6. YARN-4781.005.branch-2.patch
          26 kB
          Eric Payne

        Activity

          People

            epayne Eric Payne
            leftnoteasy Wangda Tan
            Votes:
            2 Vote for this issue
            Watchers:
            16 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: