Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9327

Improve synchronisation in ProtoUtils#convertToProtoFormat block

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0, 3.2.1, 3.1.3
    • Component/s: None
    • Labels:
      None

      Description

        public static synchronized ResourceProto convertToProtoFormat(Resource r) {
          return ResourcePBImpl.getProto(r);
        }
      
      "IPC Server handler 41 on 23764" #324 daemon prio=5 os_prio=0 tid=0x00007f181de72800 nid=0x222 waiting for monitor entry [0x00007ef153dad000]
         java.lang.Thread.State: BLOCKED (on object monitor)
      	at org.apache.hadoop.yarn.api.records.impl.pb.ProtoUtils.convertToProtoFormat(ProtoUtils.java:404)
      	- waiting to lock <0x00007ef2d8bcf6d8> (a java.lang.Class for org.apache.hadoop.yarn.api.records.impl.pb.ProtoUtils)
      	at org.apache.hadoop.yarn.api.records.impl.pb.NodeReportPBImpl.convertToProtoFormat(NodeReportPBImpl.java:315)
      	at org.apache.hadoop.yarn.api.records.impl.pb.NodeReportPBImpl.mergeLocalToBuilder(NodeReportPBImpl.java:262)
      	at org.apache.hadoop.yarn.api.records.impl.pb.NodeReportPBImpl.mergeLocalToProto(NodeReportPBImpl.java:289)
      	at org.apache.hadoop.yarn.api.records.impl.pb.NodeReportPBImpl.getProto(NodeReportPBImpl.java:228)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl.convertToProtoFormat(AllocateResponsePBImpl.java:844)
      	- locked <0x00007f0fed968a30> (a org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl.access$500(AllocateResponsePBImpl.java:72)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl$7$1.next(AllocateResponsePBImpl.java:810)
      	- locked <0x00007f0fed96f500> (a org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl$7$1)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl$7$1.next(AllocateResponsePBImpl.java:799)
      	at com.google.protobuf.AbstractMessageLite$Builder.checkForNullValues(AbstractMessageLite.java:336)
      	at com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:323)
      	at org.apache.hadoop.yarn.proto.YarnServiceProtos$AllocateResponseProto$Builder.addAllUpdatedNodes(YarnServiceProtos.java:13810)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl.mergeLocalToBuilder(AllocateResponsePBImpl.java:158)
      	- locked <0x00007f0fed968a30> (a org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl.mergeLocalToProto(AllocateResponsePBImpl.java:198)
      	- eliminated <0x00007f0fed968a30> (a org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl.getProto(AllocateResponsePBImpl.java:103)
      	- locked <0x00007f0fed968a30> (a org.apache.hadoop.yarn.api.protocolrecords.impl.pb.AllocateResponsePBImpl)
      	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationMasterProtocolPBServiceImpl.allocate(ApplicationMasterProtocolPBServiceImpl.java:61)
      	at org.apache.hadoop.yarn.proto.ApplicationMasterProtocol$ApplicationMasterProtocolService$2.callBlockingMethod(ApplicationMasterProtocol.java:99)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:524)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
      	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:878)
      	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:824)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2684)

      Seems synchronization is not required here.

        Attachments

        1. YARN-9327.003.patch
          2 kB
          Bibin Chundatt
        2. YARN-9327.002.patch
          2 kB
          Bibin Chundatt
        3. YARN-9327.001.patch
          0.8 kB
          Bibin Chundatt

          Activity

            People

            • Assignee:
              bibinchundatt Bibin Chundatt
              Reporter:
              bibinchundatt Bibin Chundatt
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: