Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8085

ResourceProfilesManager should be set in RMActiveServiceContext

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 3.1.0
    • capacityscheduler
    • None
    • Reviewed

    Description

      We submited a distributed shell application after RM failover and back to active, then got NPE error in RM log:

      java.lang.NullPointerException
              at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getResourceProfiles(ClientRMService.java:1814)
              at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getResourceProfiles(ApplicationClientProtocolPBServiceImpl.java:657)
              at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:617)
              at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
              at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
              at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:422)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)
      

      The cause is that currently resourceProfilesManager is not transferred to new RMContext instance in RMContext#resetRMContext. We should do this transfer to fix this error.

      @@ -1488,6 +1488,10 @@ private void resetRMContext() {
           // transfer service context to new RM service Context
           rmContextImpl.setServiceContext(rmContext.getServiceContext());
      
      +    // transfer resource profiles manager
      +    rmContextImpl
      +        .setResourceProfilesManager(rmContext.getResourceProfilesManager());
      +
           // reset dispatcher
           Dispatcher dispatcher = setupDispatcher();
           ((Service) dispatcher).init(this.conf);
      

      Attachments

        1. YARN-8085.001.patch
          3 kB
          Tao Yang
        2. YARN-8085.002.patch
          6 kB
          Tao Yang
        3. YARN-8085.003.patch
          7 kB
          Tao Yang
        4. YARN-8085.004.patch
          8 kB
          Tao Yang

        Activity

          People

            Tao Yang Tao Yang
            Tao Yang Tao Yang
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: