Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-397 RM Scheduler api enhancements
  3. YARN-1042

add ability to specify affinity/anti-affinity in container requests

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.0.0-alpha1
    • None
    • resourcemanager
    • None

    Description

      container requests to the AM should be able to request anti-affinity to ensure that things like Region Servers don't come up on the same failure zones.

      Similarly, you may be able to want to specify affinity to same host or rack without specifying which specific host/rack. Example: bringing up a small giraph cluster in a large YARN cluster would benefit from having the processes in the same rack purely for bandwidth reasons.

      Attachments

        1. YARN-1042-global-scheduling.poc.1.patch
          60 kB
          Wangda Tan
        2. YARN-1042.002.patch
          57 kB
          Weiwei Yang
        3. YARN-1042.001.patch
          54 kB
          Weiwei Yang
        4. YARN-1042-design-doc.pdf
          68 kB
          Weiwei Yang
        5. YARN-1042-demo.patch
          15 kB
          Junping Du

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue YARN-624 Support gang scheduling in the AM RM protocol

          • Major - Major loss of function.
          • Open
          is cloned by

          New Feature - A new feature of the product, which has yet to be developed. YARN-5907 [Umbrella] [YARN-1042] add ability to specify affinity/anti-affinity in container requests

          • Major - Major loss of function.
          • Open
          is depended upon by

          New Feature - A new feature of the product, which has yet to be developed. YARN-896 Roll up for long-lived services in YARN

          • Major - Major loss of function.
          • Open

          Wish - General wishlist item. SAMZA-74 Use anti-affinity in YARN container requests

          • Major - Major loss of function.
          • Open
          is duplicated by

          Sub-task - The sub-task of the issue YARN-2969 allocate resource on different nodes for task

          • Major - Major loss of function.
          • Resolved
          is related to

          Task - A task that needs to be done. SLIDER-82 Support ANTI_AFFINITY_REQUIRED option

          • Major - Major loss of function.
          • Resolved
          relates to

          Bug - A problem which impairs or prevents the functions of the product. YARN-2027 YARN ignores host-specific resource requests

          • Major - Major loss of function.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. TWILL-87 Adding Container Placement control (Placement Policy API)

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. YARN-4879 Enhance Allocate Protocol to Identify Requests Explicitly

          • Major - Major loss of function.
          • Resolved
          requires

          New Feature - A new feature of the product, which has yet to be developed. YARN-5139 [Umbrella] Move YARN scheduler towards global scheduler

          • Major - Major loss of function.
          • Open

          Activity

            People

              leftnoteasy Wangda Tan
              stevel@apache.org Steve Loughran
              Votes:
              5 Vote for this issue
              Watchers:
              61 Start watching this issue

              Dates

                Created:
                Updated: