Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3409

Support Node Attribute functionality

          Details

          • Type: New Feature
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 3.2.0
          • Component/s: api, client, RM
          • Labels:
            None
          • Release Note:
            With this feature task, Node Attributes is supported in YARN which will help user's to effectively use resources and assign to applications based on characteristics of each node's in the cluster.

            Description

            Specify only one label for each node (IAW, partition a cluster) is a way to determinate how resources of a special set of nodes could be shared by a group of entities (like teams, departments, etc.). Partitions of a cluster has following characteristics:

            • Cluster divided to several disjoint sub clusters.
            • ACL/priority can apply on partition (Only market team / marke team has priority to use the partition).
            • Percentage of capacities can apply on partition (Market team has 40% minimum capacity and Dev team has 60% of minimum capacity of the partition).

            Attributes are orthogonal to partition, they’re describing features of node’s hardware/software just for affinity. Some example of attributes:

            • glibc version
            • JDK version
            • Type of CPU (x86_64/i686)
            • Type of OS (windows, linux, etc.)

            With this, application can be able to ask for resource has (glibc.version >= 2.20 && JDK.version >= 8u20 && x86_64).

              Attachments

              1. Constraint-Node-Labels-Requirements-Design-doc_v1.pdf
                153 kB
                Naganarasimha G R
              2. YARN-3409.WIP.001.patch
                91 kB
                Naganarasimha G R
              3. 3409-apiChanges_v2.pdf (4).pdf
                71 kB
                Naganarasimha G R
              4. Node-Attributes-Requirements-Design-doc_v2.pdf
                143 kB
                Naganarasimha G R

                Issue Links

                Dependency

                Improvement - An improvement or enhancement to an existing feature or task. YARN-8670 Support scheduling request for SLS input and attributes for Node in SLS

                • Major - Major loss of function.
                • Open
                is depended upon by

                Sub-task - The sub-task of the issue YARN-6720 Support updating FPGA related constraint node label after FPGA device re-configuration

                • Major - Major loss of function.
                • Open

                Improvement - An improvement or enhancement to an existing feature or task. YARN-8766 Support for Node Attribute Functionality, Phase 2

                • Major - Major loss of function.
                • Open
                is related to

                New Feature - A new feature of the product, which has yet to be developed. YARN-5683 Support specifying storage type for per-application local dirs

                • Major - Major loss of function.
                • Patch Available

                Task - A task that needs to be done. YARN-2492 (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests

                • Major - Major loss of function.
                • Open
                relates to

                Sub-task - The sub-task of the issue YARN-6346 Expose API in ApplicationSubmissionContext to specify placement constraints

                • Major - Major loss of function.
                • Resolved
                There are no Sub-Tasks for this issue.

                  Activity

                    People

                    • Assignee:
                      Naganarasimha Naganarasimha G R
                      Reporter:
                      leftnoteasy Wangda Tan
                    • Votes:
                      2 Vote for this issue
                      Watchers:
                      47 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: