Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: mrv2, resourcemanager
    • Labels:
      None
    • Environment:

      Description

      When I try to register my ApplicationMaster with YARN's RM, it fails.

      In my ApplicationMaster's logs:

      Exception in thread "main" java.lang.reflect.UndeclaredThrowableException
      at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:108)
      at kafka.yarn.util.ApplicationMasterHelper.registerWithResourceManager(YarnHelper.scala:48)
      at kafka.yarn.ApplicationMaster$.main(ApplicationMaster.scala:32)
      at kafka.yarn.ApplicationMaster.main(ApplicationMaster.scala)
      Caused by: com.google.protobuf.ServiceException: java.lang.NullPointerException: java.lang.NullPointerException
      at org.apache.hadoop.yarn.proto.ClientRMProtocol$ClientRMProtocolService$2.getRequestPrototype(ClientRMProtocol.java:186)
      at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:323)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1489)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1485)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1483)

      at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:130)
      at $Proxy6.registerApplicationMaster(Unknown Source)
      at org.apache.hadoop.yarn.api.impl.pb.client.AMRMProtocolPBClientImpl.registerApplicationMaster(AMRMProtocolPBClientImpl.java:101)
      ... 3 more
      Caused by: java.lang.NullPointerException: java.lang.NullPointerException
      at org.apache.hadoop.yarn.proto.ClientRMProtocol$ClientRMProtocolService$2.getRequestPrototype(ClientRMProtocol.java:186)
      at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:323)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1489)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1485)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1483)

      at org.apache.hadoop.ipc.Client.call(Client.java:1084)
      at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:127)
      ... 5 more

      In the ResourceManager's logs:

      2011-09-20 15:11:20,973 INFO ipc.Server (Server.java:run(1497)) - IPC Server handler 2 on 8040, call: org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$ProtoSpecificRequestWritable@455dd32a from 127.0.0.1:33793, error:
      java.lang.NullPointerException
      at org.apache.hadoop.yarn.proto.ClientRMProtocol$ClientRMProtocolService$2.getRequestPrototype(ClientRMProtocol.java:186)
      at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:323)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1489)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1485)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1483)

      My registration code:

      val appId = args(0).toInt
      val attemptId = args(1).toInt
      val timestamp = args(2).toLong

      // these are our application master's parameters
      val streamerClass = args(3)
      val tasks = args(4).toInt

      // TODO log params here

      // start the application master helper
      val conf = new Configuration
      val applicationMasterHelper = new ApplicationMasterHelper(appId, attemptId, timestamp, conf)
      .registerWithResourceManager

      .....

      val rpc = YarnRPC.create(conf)
      val appId = Records.newRecord(classOf[ApplicationId])
      val appAttemptId = Records.newRecord(classOf[ApplicationAttemptId])
      val rmAddress = NetUtils.createSocketAddr(conf.get(YarnConfiguration.RM_ADDRESS, YarnConfiguration.DEFAULT_RM_ADDRESS))
      val resourceManager = rpc.getProxy(classOf[AMRMProtocol], rmAddress, conf).asInstanceOf[AMRMProtocol]
      var requestId = 0

      appId.setClusterTimestamp(lTimestamp)
      appId.setId(iAppId)
      appAttemptId.setApplicationId(appId)
      appAttemptId.setAttemptId(iAppAttemptId)

      def registerWithResourceManager(): ApplicationMasterHelper =

      { val req = Records.newRecord(classOf[RegisterApplicationMasterRequest]) req.setApplicationAttemptId(appAttemptId) // TODO not sure why these are blank- This is how spark does it req.setHost("") req.setRpcPort(1) req.setTrackingUrl("") resourceManager.registerApplicationMaster(req) this }

      My params are receiving the proper app/attempt/cluster timestamps:

      app - 1
      attempt - 1
      timestamp - 1316556657998

      1. MAPREDUCE-3053-20110922.txt
        4 kB
        Vinod Kumar Vavilapalli

        Activity

        No work has yet been logged on this issue.

          People

          • Assignee:
            Vinod Kumar Vavilapalli
            Reporter:
            Chris Riccomini
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development