Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6592

[Umbrella] Rich placement constraints in YARN

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0
    • None
    • None

    Description

      This JIRA consolidates the efforts of YARN-5468 and YARN-4902.
      It adds support for rich placement constraints to YARN, such as affinity and anti-affinity between allocations within the same or across applications.

      Attachments

        Issue Links

          is depended upon by

          Sub-task - The sub-task of the issue YARN-7142 Support placement policy in yarn native services

          • Major - Major loss of function.
          • Resolved

          Sub-task - The sub-task of the issue YARN-7441 Support placement policy in yarn service

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-5468 Scheduling of long-running applications

          • Major - Major loss of function.
          • Resolved
          is related to

          Sub-task - The sub-task of the issue YARN-5906 Update AppSchedulingInfo to use SchedulingPlacementSet

          • Major - Major loss of function.
          • Resolved

          New Feature - A new feature of the product, which has yet to be developed. YARN-7812 Improvements to Rich Placement Constraints in YARN

          • Major - Major loss of function.
          • Resolved
          1.
          		 [API] Introduce Placement Constraint object Sub-task Resolved Konstantinos Karanasos
          2.
          		 [API] Introduce SchedulingRequest object Sub-task Resolved Konstantinos Karanasos
          3.
          		 [API] Add Placement Constraints at the application level Sub-task Resolved Arun Suresh
          4.
          		 [API] Add SchedulingRequest to the AllocateRequest Sub-task Resolved Panagiotis Garefalakis
          5.
          		 Introduce AllocationTagsManager to associate allocation tags to nodes Sub-task Resolved Wangda Tan
          6.
          		 Modifications to the ResourceScheduler to support SchedulingRequests Sub-task Resolved Arun Suresh
          7.
          		 API and interface modifications for placement constraint processor Sub-task Resolved Arun Suresh
          8.
          		 Rack cardinality support for AllocationTagsManager Sub-task Resolved Panagiotis Garefalakis
          9.
          		 Introduce Placement Constraint Manager module Sub-task Resolved Konstantinos Karanasos
          10.
          		 Add Processor Framework for Rich Placement Constraints Sub-task Resolved Arun Suresh
          11.
          		 Implement Basic algorithm for constraint based placement Sub-task Resolved Panagiotis Garefalakis
          12.
          		 Expose canSatisfyConstraints utility function to validate a placement against a constraint Sub-task Resolved Panagiotis Garefalakis
          13.
          		 Double-check placement constraints in scheduling phase before actual allocation is made Sub-task Resolved Weiwei Yang
          14.
          		 Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases Sub-task Resolved Arun Suresh
          15.
          		 AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects Sub-task Resolved Arun Suresh
          16.
          		 Remove SELF from TargetExpression type Sub-task Resolved Konstantinos Karanasos
          17.
          		 Allow DistributedShell to take a placement specification for containers it wants to launch Sub-task Resolved Arun Suresh
          18.
          		 Miscellaneous fixes to the PlacementProcessor Sub-task Resolved Arun Suresh
          19.
          		 Support anti-affinity constraint via AppPlacementAllocator Sub-task Resolved Wangda Tan
          20.
          		 Allow Constraints specified in the SchedulingRequest to override application level constraints Sub-task Resolved Weiwei Yang
          21.
          		 Add examples for placement constraints usage in applications Sub-task Closed Panagiotis Garefalakis
          22.
          		 Add RMContainer recovery test to verify tag population in the AllocationTagsManager Sub-task Resolved Panagiotis Garefalakis
          23.
          		 Factor out management of temp tags from AllocationTagsManager Sub-task Resolved Arun Suresh
          24.
          		 Add validation step to ensure constraints are not violated due to order in which a request is processed Sub-task Resolved Arun Suresh
          25.
          		 Display allocation tags in RM web UI and expose same through REST API Sub-task Resolved Weiwei Yang
          26.
          		 Assume intra-app anti-affinity as default for scheduling request inside AppPlacementAllocator Sub-task Resolved Wangda Tan
          27.
          		 Fix jenkins issues of YARN-6592 branch Sub-task Resolved Sunil G
          28.
          		 Fix Cluster metrics when placement processor is enabled Sub-task Resolved Arun Suresh
          29.
          		 Documentation for Placement Constraints Sub-task Resolved Konstantinos Karanasos
          30.
          		 Merge work for YARN-6592 Sub-task Closed Sunil G

          Activity

            People

              Unassigned Unassigned
              kkaranasos Konstantinos Karanasos
              Votes:
              0 Vote for this issue
              Watchers:
              28 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: