[YARN-7069] [Umbrella] YARN Resource Profiles phase#2 enhancements

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: nodemanager, resourcemanager
• Assignee: Unassigned
• Reporter: Sunil G
• Created: 22/Aug/17 07:07
• Updated: 18/Jul/18 22:25

Description

This ticket to is to track further enhancements to resource profile work happening in YARN-3926

Issue Links

• is blocked by: YARN-7314 Multiple Resource manager test cases failing
• is related to: YARN-8200 Backport resource types/GPU features to branch-3.0/branch-2
• relates to: YARN-3926 [Umbrella] Extend the YARN resource model for easier resource-type management and profiles

Sub-Tasks

• Add support for increase and decrease of container resources with resource profiles: Patch Available
• Update CapacitySchedulerConfiguration minimum and maximum calculations to consider all resource types: Patch Available
• Update web UIs to reflect multiple resource types: Open
• Add support for dynamic resource updates with multiple resource types: Open
• Add support for resource profiles in MapReduce: Patch Available
• ResourceTypes should be renamed: Open
• Cluster Metrics API should report resource types information: Open
• Cluster Node API and Cluster Nodes API should report resource types information: Patch Available
• TestResourceUtils to be moved to hadoop-yarn-api package: Open
• [Resource Profiles] getMaximumAllocationPerQueue to account for all resources: Patch Available
• Cleaning up the usage of Resources and ResourceCalculator: Patch Available
• Node manager should allow resource units to be lower cased: Open
• Resource configs not parsed correctly in MapReduce for resources with dot: Patch Available
• RM gives YARNFeatureNotEnabledException even when resource profile feature is not enabled: Patch Available
• Adding profile capability in resourceReq in LocalityMulticastAMRMProxyPolicy: Patch Available
• Fair Scheduler stops scheduling when a queue is configured only CPU and memory: Patch Available