[YARN-8278] DistributedScheduling is not working in HA

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.0.0, 3.1.0
• Fix Version/s: 3.2.0, 3.1.1, 3.0.3
• Target Version/s: 3.1.1
• Hadoop Flags: Reviewed
• Assignee: Bibin Chundatt
• Reporter: Bibin Chundatt
• Created: 11/May/18 11:27
• Updated: 15/May/18 14:29
• Resolved: 15/May/18 10:58

Description

Configured HA Cluster and submit application with distributed scheduling configuration enabled

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.yarn.exceptions.YarnRuntimeException): java.lang.IllegalArgumentException: ResourceManager does not support this protocol
        at org.apache.hadoop.yarn.server.nodemanager.amrmproxy.DefaultRequestInterceptor.init(DefaultRequestInterceptor.java:91)
        at org.apache.hadoop.yarn.server.nodemanager.amrmproxy.AbstractRequestInterceptor.init(AbstractRequestInterceptor.java:82)
        at org.apache.hadoop.yarn.server.nodemanager.scheduler.DistributedScheduler.init(DistributedScheduler.java:89)
        at org.apache.hadoop.yarn.server.nodemanager.amrmproxy.AMRMProxyService.initializePipeline(AMRMProxyService.java:450)
        at org.apache.hadoop.yarn.server.nodemanager.amrmproxy.AMRMProxyService.processApplicationStartRequest(AMRMProxyService.java:369)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl.startContainers(ContainerManagerImpl.java:942)
        at org.apache.hadoop.yarn.api.impl.pb.service.ContainerManagementProtocolPBServiceImpl.startContainers(ContainerManagementProtocolPBServiceImpl.java:101)
        at org.apache.hadoop.yarn.proto.ContainerManagementProtocol$ContainerManagementProtocolService$2.callBlockingMethod(ContainerManagementProtocol.java:223)
        at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:523)
        at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:869)
        at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:815)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1682)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2675)

ServerRMProxy#checkAllowedProtocols should allow DistributedSchedulingAMProtocol

  public void checkAllowedProtocols(Class<?> protocol) {
    Preconditions.checkArgument(
        protocol.isAssignableFrom(ResourceTracker.class),
        "ResourceManager does not support this protocol");
  }

Issue Links

• is related to: YARN-4742 [Umbrella] Enhancements to Distributed Scheduling