[YARN-2492] (Clone of YARN-796) Allow for (admin) labels on nodes and resource-requests

• Type: Task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: api, client, resourcemanager
• Assignee: Unassigned
• Reporter: Wangda Tan
• Created: 03/Sep/14 08:47
• Updated: 25/Oct/19 20:25

Description

Since YARN-796 is a sub JIRA of YARN-397, this JIRA is used to create and track sub tasks and attach split patches for YARN-796.

Let's still keep over-all discussions on YARN-796.

Issue Links

• is cloned by: YARN-796 Allow for (admin) labels on nodes and resource-requests
• relates to: YARN-3894 RM startup should fail for wrong CS xml NodeLabel capacity configuration
• relates to: YARN-3971 Skip RMNodeLabelsManager#checkRemoveFromClusterNodeLabelsOfQueue on nodelabel recovery
• relates to: YARN-4454 NM to nodelabel mapping going wrong after RM restart
• relates to: YARN-4106 NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
• relates to: YARN-4176 Resync NM nodelabels with RM periodically for distributed nodelabels
• relates to: YARN-5788 Apps not activiated and AM limit resource in UI and REST not updated after -replaceLabelsOnNode
• relates to: MAPREDUCE-6304 Specifying node labels when submitting MR jobs
• relates to: YARN-3409 Support Node Attribute functionality
• relates to: YARN-4652 Skip label configuration and calculation of queue if nodelabel disabled
• relates to: YARN-4582 Label-related invalid resource request exception should be able to properly handled by application
• relates to: YARN-4465 SchedulerUtils#validateRequest for Label check should happen only when nodelabel enabled

Sub-Tasks

• Fair scheduler should support strict node labels: Patch Available
• Respect labels in preemption policy of fair scheduler: Open
• Support continuously looking reserved container with node labels: Open
• Support for disabling the Centralized NodeLabel validation in Distributed Node Label Configuration setup: Open
• Support load command from admin [Helps to load big set of labels]: Open
• Duplicate check for partitioned request on partitioned node: Open
• Error handling on NM reporting invalid NodeLabels in distributed Node Label configuration: Open
• Support intra-queue non-exclusive container allocation within a LeafQueue: Open
• We should be able to remove node-label if there's no queue can use it.: Open
• Scheduler Web Ui shows max capacity for the queue is 100% but when we submit application doesnt get assigned: Open
• Pluggable sharing policy for Partition Node Label resources: Open
• Add documentation for the Node Label features supported in 2.6: Open
• Fair Scheduler: respect node labels at resource request level: Open
• Add configuration to initialize RM with configured labels.: Patch Available