Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-6640

AM heartbeat stuck when responseId overflows MAX_INT

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.2
    • Component/s: None
    • Labels:
      None

      Description

      The current code in ApplicationMasterService:
      if ((request.getResponseId() + 1) == lastResponse.getResponseId())

      {/* old heartbeat */ return lastResponse;}

      else if (request.getResponseId() + 1 < lastResponse.getResponseId())

      { throw ... }

      process the heartbeat...

      When a heartbeat comes in, in usual case we are expecting request.getResponseId() == lastResponse.getResponseId(). The “if“ is for the duplicate heartbeat that’s one step old, the “else if” is to throw and complain for heartbeats more than two steps old, otherwise we accept the new heartbeat and process it.

      So the bug is: when lastResponse.getResponseId() == MAX_INT, the newest heartbeat comes in with responseId == MAX_INT. However reponseId + 1 will be MIN_INT, and we will fall into the “else if” case and RM will throw. Then we are stuck here…

        Attachments

        1. YARN-6640.v2.patch
          9 kB
          Botong Huang
        2. YARN-6640.v1.patch
          10 kB
          Botong Huang

          Issue Links

            Activity

              People

              • Assignee:
                botong Botong Huang
                Reporter:
                botong Botong Huang
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: