[YARN-8085] ResourceProfilesManager should be set in RMActiveServiceContext

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 3.1.0
• Component/s: capacityscheduler
• Target Version/s: 3.1.0
• Hadoop Flags: Reviewed
• Assignee: Tao Yang
• Reporter: Tao Yang
• Created: 29/Mar/18 04:49
• Updated: 30/Mar/18 02:47
• Resolved: 29/Mar/18 16:12

Description

We submited a distributed shell application after RM failover and back to active, then got NPE error in RM log:

java.lang.NullPointerException
        at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getResourceProfiles(ClientRMService.java:1814)
        at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getResourceProfiles(ApplicationClientProtocolPBServiceImpl.java:657)
        at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:617)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)

The cause is that currently resourceProfilesManager is not transferred to new RMContext instance in RMContext#resetRMContext. We should do this transfer to fix this error.

@@ -1488,6 +1488,10 @@ private void resetRMContext() {
     // transfer service context to new RM service Context
     rmContextImpl.setServiceContext(rmContext.getServiceContext());

+    // transfer resource profiles manager
+    rmContextImpl
+        .setResourceProfilesManager(rmContext.getResourceProfilesManager());
+
     // reset dispatcher
     Dispatcher dispatcher = setupDispatcher();
     ((Service) dispatcher).init(this.conf);