Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-5864

YARN Capacity Scheduler - Queue Priorities

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently, Capacity Scheduler at every parent-queue level uses relative used-capacities of the chil-queues to decide which queue can get next available resource first.

      For example,

      • Q1 & Q2 are child queues under queueA
      • Q1 has 20% of configured capacity, 5% of used-capacity and
      • Q2 has 80% of configured capacity, 8% of used-capacity.

      In the situation, the relative used-capacities are calculated as below

      • Relative used-capacity of Q1 is 5/20 = 0.25
      • Relative used-capacity of Q2 is 8/80 = 0.10

      In the above example, per today’s Capacity Scheduler’s algorithm, Q2 is selected by the scheduler first to receive next available resource.

      Simply ordering queues according to relative used-capacities sometimes causes a few troubles because scarce resources could be assigned to less-important apps first.

      1. Latency sensitivity: This can be a problem with latency sensitive applications where waiting till the ‘other’ queue gets full is not going to cut it. The delay in scheduling directly reflects in the response times of these applications.
      2. Resource fragmentation for large-container apps: Today’s algorithm also causes issues with applications that need very large containers. It is possible that existing queues are all within their resource guarantees but their current allocation distribution on each node may be such that an application which needs large container simply cannot fit on those nodes.
        Services:
      3. The above problem (2) gets worse with long running applications. With short running apps, previous containers may eventually finish and make enough space for the apps with large containers. But with long running services in the cluster, the large containers’ application may never get resources on any nodes even if its demands are not yet met.
      4. Long running services are sometimes more picky w.r.t placement than normal batch apps. For example, for a long running service in a separate queue (say queue=service), during peak hours it may want to launch instances on 50% of the cluster nodes. On each node, it may want to launch a large container, say 200G memory per container.

        Attachments

        1. YARN-5864.poc-0.patch
          19 kB
          Wangda Tan
        2. YARN-CapacityScheduler-Queue-Priorities-design-v1.pdf
          178 kB
          Wangda Tan
        3. YARN-5864.001.patch
          125 kB
          Wangda Tan
        4. YARN-5864.002.patch
          156 kB
          Wangda Tan
        5. YARN-5864.003.patch
          165 kB
          Wangda Tan
        6. YARN-5864.004.patch
          175 kB
          Wangda Tan
        7. YARN-5864.005.patch
          176 kB
          Wangda Tan
        8. YARN-5864.006.patch
          176 kB
          Wangda Tan
        9. YARN-5864-preemption-performance-report.pdf
          200 kB
          Wangda Tan
        10. YARN-5864-usage-doc.html
          16 kB
          Wangda Tan
        11. YARN-5864.007.patch
          176 kB
          Wangda Tan
        12. YARN-5864.branch-2.007.patch
          176 kB
          Wangda Tan
        13. YARN-5864.branch-2.007_2.patch
          176 kB
          Wangda Tan
        14. YARN-5864.branch-2.008.patch
          175 kB
          Wangda Tan
        There are no Sub-Tasks for this issue.

          Activity

            People

            • Assignee:
              leftnoteasy Wangda Tan
              Reporter:
              leftnoteasy Wangda Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              21 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: