Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6592

[Umbrella] Rich placement constraints in YARN

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      This JIRA consolidates the efforts of YARN-5468 and YARN-4902.
      It adds support for rich placement constraints to YARN, such as affinity and anti-affinity between allocations within the same or across applications.

        Attachments

          Issue Links

          1.
          [API] Introduce Placement Constraint object Sub-task Resolved Konstantinos Karanasos
          2.
          [API] Introduce SchedulingRequest object Sub-task Resolved Konstantinos Karanasos
          3.
          [API] Add Placement Constraints at the application level Sub-task Resolved Arun Suresh
          4.
          [API] Add SchedulingRequest to the AllocateRequest Sub-task Resolved Panagiotis Garefalakis
          5.
          Introduce AllocationTagsManager to associate allocation tags to nodes Sub-task Resolved Wangda Tan
          6.
          Modifications to the ResourceScheduler to support SchedulingRequests Sub-task Resolved Arun Suresh
          7.
          API and interface modifications for placement constraint processor Sub-task Resolved Arun Suresh
          8.
          Rack cardinality support for AllocationTagsManager Sub-task Resolved Panagiotis Garefalakis
          9.
          Introduce Placement Constraint Manager module Sub-task Resolved Konstantinos Karanasos
          10.
          Add Processor Framework for Rich Placement Constraints Sub-task Resolved Arun Suresh
          11.
          Implement Basic algorithm for constraint based placement Sub-task Resolved Panagiotis Garefalakis
          12.
          Expose canSatisfyConstraints utility function to validate a placement against a constraint Sub-task Resolved Panagiotis Garefalakis
          13.
          Double-check placement constraints in scheduling phase before actual allocation is made Sub-task Resolved Weiwei Yang
          14.
          Add container tags to ContainerTokenIdentifier, api.Container and NMContainerStatus to handle all recovery cases Sub-task Resolved Arun Suresh
          15.
          AMRMClient Changes to use the PlacementConstraint and SchcedulingRequest objects Sub-task Resolved Arun Suresh
          16.
          Remove SELF from TargetExpression type Sub-task Resolved Konstantinos Karanasos
          17.
          Allow DistributedShell to take a placement specification for containers it wants to launch Sub-task Resolved Arun Suresh
          18.
          Miscellaneous fixes to the PlacementProcessor Sub-task Resolved Arun Suresh
          19.
          Support anti-affinity constraint via AppPlacementAllocator Sub-task Resolved Wangda Tan
          20.
          Allow Constraints specified in the SchedulingRequest to override application level constraints Sub-task Resolved Weiwei Yang
          21.
          Add examples for placement constraints usage in applications Sub-task Closed Panagiotis Garefalakis
          22.
          Add RMContainer recovery test to verify tag population in the AllocationTagsManager Sub-task Resolved Panagiotis Garefalakis
          23.
          Factor out management of temp tags from AllocationTagsManager Sub-task Resolved Arun Suresh
          24.
          Add validation step to ensure constraints are not violated due to order in which a request is processed Sub-task Resolved Arun Suresh
          25.
          Display allocation tags in RM web UI and expose same through REST API Sub-task Resolved Weiwei Yang
          26.
          Assume intra-app anti-affinity as default for scheduling request inside AppPlacementAllocator Sub-task Resolved Wangda Tan
          27.
          Fix jenkins issues of YARN-6592 branch Sub-task Resolved Sunil G
          28.
          Fix Cluster metrics when placement processor is enabled Sub-task Resolved Arun Suresh
          29.
          Documentation for Placement Constraints Sub-task Resolved Konstantinos Karanasos
          30.
          Merge work for YARN-6592 Sub-task Closed Sunil G

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kkaranasos Konstantinos Karanasos
              • Votes:
                0 Vote for this issue
                Watchers:
                26 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: