[YARN-6152] Used queue percentage not accurate in UI for 2.7 and below when using DominantResourceCalculator

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.3
• Fix Version/s: 2.7.4
• Target Version/s: 2.7.4
• Assignee: Jonathan Hung
• Reporter: Jonathan Hung
• Created: 06/Feb/17 22:29
• Updated: 14/Oct/19 15:37
• Resolved: 07/Feb/17 17:40

Description

YARN-4751 adds the getUsedCapacity and getAbsoluteUsedCapacity methods to AbstractCSQueue which is used to display queue usage in UI for branch-2.7 and below. However if there is more than one partition in the cluster, with different dominant resources, then queue usage may not be displayed as expected.

Contrived example: Default partition has <90GB, 10vcores>, and "test" partition has <10GB, 90vcores>. root queue in default partition uses <30GB, 10vcores>. Here we expect queue usage to be 100 since it is using all vcores in default partition. But the displayed usage will be (30GB/100GB)/(90GB/100GB) = 33%.

Issue Links

• is broken by: YARN-4751 In 2.7, Labeled queue usage not shown properly in capacity scheduler UI
• is related to: YARN-6154 Unit tests for DominantResourceCalculator queue used capacity