[YARN-7575] NPE in scheduler UI when max-capacity is not configured

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0
• Component/s: capacity scheduler
• Hadoop Flags: Reviewed
• Assignee: Sunil G
• Reporter: Eric Payne
• Created: 28/Nov/17 17:32
• Updated: 14/Oct/19 15:38
• Resolved: 29/Nov/17 09:22

Description

I encountered the following while reviewing and testing branch YARN-5881.

The design document from YARN-5881 says that for max-capacity:

3) For each queue, we require:
a) if max-resource not set, it automatically set to parent.max-resource

When I try leaving blank yarn.scheduler.capacity.< queue-path>.maximum-capacity, the RMUI scheduler page refuses to render. It looks like it's in CapacitySchedulerPage$ LeafQueueInfoBlock:

2017-11-28 11:29:16,974 [qtp43473566-220] ERROR webapp.Dispatcher: error handling URI: /cluster/scheduler
java.lang.reflect.InvocationTargetException
...
        at org.apache.hadoop.yarn.server.resourcemanager.webapp.CapacitySchedulerPage$LeafQueueInfoBlock.renderQueueCapacityInfo(CapacitySchedulerPage.java:164)
        at org.apache.hadoop.yarn.server.resourcemanager.webapp.CapacitySchedulerPage$LeafQueueInfoBlock.renderLeafQueueInfoWithoutParition(CapacitySchedulerPage.java:129)

Also... A job will run in the leaf queue with no max capacity set and it will grow to the max capacity of the cluster, but if I add resources to the node, the job won't grow any more even though it has pending resources.