Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9610

HeartbeatCallBack int FederationInterceptor clear AMRMToken in response from UAM should before add to aysncResponseSink

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 3.2.0
    • Fix Version/s: None
    • Component/s: amrmproxy, federation
    • Labels:
      None

      Description

      in federation, `allocate` is async. the response from RM is cached in `asyncResponseSink`.

      the final allocate response is merged from all RMs allocate response. merge will throw exception when AMRMToken from UAM response is not null.

      But set AMRMToken from UAM response to null is not in the scope of lock. so there will be a change merge see that  AMRMToken from UAM response is not null.

      so we should clear the token before add response to asyncResponseSink

       

       

      synchronized (asyncResponseSink) {
        List<AllocateResponse> responses = null;
        if (asyncResponseSink.containsKey(subClusterId)) {
          responses = asyncResponseSink.get(subClusterId);
        } else {
          responses = new ArrayList<>();
          asyncResponseSink.put(subClusterId, responses);
        }
        responses.add(response);
        // Notify main thread about the response arrival
        asyncResponseSink.notifyAll();
      }
      ...
      if (this.isUAM && response.getAMRMToken() != null) {
        Token<AMRMTokenIdentifier> newToken = ConverterUtils
            .convertFromYarn(response.getAMRMToken(), (Text) null);
        // Do not further propagate the new amrmToken for UAM
        response.setAMRMToken(null);
      ...

        Attachments

        1. YARN-9610.patch.1
          8 kB
          Morty Zhong
        2. YARN-9610.patch.2
          8 kB
          Morty Zhong

          Activity

            People

            • Assignee:
              Cedar Morty Zhong
              Reporter:
              Cedar Morty Zhong
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: