[YARN-10226] NPE in Capacity Scheduler while using %primary_group queue mapping

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.4.0
• Component/s: capacity scheduler
• Target Version/s: 3.3.0
• Hadoop Flags: Reviewed
• Assignee: Peter Bacsko
• Reporter: Peter Bacsko
• Created: 08/Apr/20 13:56
• Updated: 14/Apr/20 13:02
• Resolved: 09/Apr/20 13:25

Description

If we use the following queue mapping:

u:%user:%primary_group

then we get a NPE inside ResourceManager:

2020-04-06 11:59:13,883 ERROR resourcemanager.ResourceManager (ResourceManager.java:serviceStart(881)) - Failed to load/recover state
java.lang.NullPointerException
        at java.util.concurrent.ConcurrentHashMap.get(ConcurrentHashMap.java:936)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerQueueManager.getQueue(CapacitySchedulerQueueManager.java:138)
        at org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.getContextForPrimaryGroup(UserGroupMappingPlacementRule.java:163)
        at org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.getPlacementForUser(UserGroupMappingPlacementRule.java:118)
        at org.apache.hadoop.yarn.server.resourcemanager.placement.UserGroupMappingPlacementRule.getPlacementForApp(UserGroupMappingPlacementRule.java:227)
        at org.apache.hadoop.yarn.server.resourcemanager.placement.PlacementManager.placeApplication(PlacementManager.java:67)
        at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.placeApplication(RMAppManager.java:827)
        at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:378)
        at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recoverApplication(RMAppManager.java:367)
        at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.recover(RMAppManager.java:594)
...

We to check if parent queue is null in UserGroupMappingPlacementRule.getContextForPrimaryGroup().

Issue Links

• is related to: YARN-10198 [managedParent].%primary_group mapping rule doesn't work after YARN-9868