[YARN-3432] Cluster metrics have wrong Total Memory when there is reserved memory on CS

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
• Component/s: capacityscheduler, resourcemanager
• Labels: oct16-easy
• Target Version/s: 2.8.0, 2.7.4
• Hadoop Flags: Reviewed
• Assignee: Brahma Reddy Battula
• Reporter: Thomas Graves
• Created: 01/Apr/15 21:16
• Updated: 19/Apr/17 02:24
• Resolved: 27/Oct/16 22:26

Description

I noticed that when reservations happen when using the Capacity Scheduler, the UI and web services report the wrong total memory.

For example. I have a 300GB of total memory in my cluster. I allocate 50 and I reserve 10. The cluster metrics for total memory get reported as 290GB.

This was broken by https://issues.apache.org/jira/browse/YARN-656 so perhaps there is a difference between fair scheduler and capacity scheduler.

Issue Links

• is broken by: YARN-656 In scheduler UI, including reserved memory in "Memory Total" can make it exceed cluster capacity.
• relates to: YARN-6491 Move totalMB and totalVirtualCores computation from ClusterMetricsInfo to QueueMetrics