Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8506

Make GetApplicationsRequestPBImpl thread safe

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0, 3.1.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When GetApplicationRequestPBImpl is used in multi-thread environment, exceptions like below will occur because we don't protect write ops.

      java.lang.ArrayIndexOutOfBoundsException
      	at java.lang.System.arraycopy(Native Method)
      	at java.util.ArrayList.addAll(ArrayList.java:613)
      	at com.google.protobuf.LazyStringArrayList.addAll(LazyStringArrayList.java:132)
      	at com.google.protobuf.LazyStringArrayList.addAll(LazyStringArrayList.java:123)
      	at com.google.protobuf.AbstractMessageLite$Builder.addAll(AbstractMessageLite.java:327)
      	at org.apache.hadoop.yarn.proto.YarnServiceProtos$GetApplicationsRequestProto$Builder.addAllApplicationTags(YarnServiceProtos.java:24450)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetApplicationsRequestPBImpl.mergeLocalToBuilder(GetApplicationsRequestPBImpl.java:100)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetApplicationsRequestPBImpl.mergeLocalToProto(GetApplicationsRequestPBImpl.java:78)
      	at org.apache.hadoop.yarn.api.protocolrecords.impl.pb.GetApplicationsRequestPBImpl.getProto(GetApplicationsRequestPBImpl.java:69)
      

      We need to make GetApplicationRequestPBImpl thread safe. We saw the issue happens frequently when RequestHedgingRMFailoverProxyProvider is being used.

        Attachments

        1. YARN-8506.002.patch
          6 kB
          Wangda Tan
        2. YARN-8506.001.patch
          6 kB
          Wangda Tan

          Activity

            People

            • Assignee:
              leftnoteasy Wangda Tan
              Reporter:
              leftnoteasy Wangda Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: