Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3730

Allow restarted NM to rejoin cluster before RM expires it

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 2.0.0-alpha
    • Fix Version/s: 0.23.2
    • Component/s: mrv2, resourcemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry.

      Description

      When a node in the RUNNING state (healthy or unhealthy) is rebooted, the resourcemanager rejects the nodemanager's registration request as a duplicate because it is convinced that the nodemanager is already running on that node. It won't allow that node to rejoin the cluster until the node expiration time elapses which is 10min+ by default. We should allow the NM to rejoin the cluster if it re-registers within the expiration timeout.

      Note that this problem occurs with NMs that are configured to specific ports. If ephemeral ports are used then a NM reboot "works" because the RM thinks the NM registration is for a new node. See the discussions in MAPREDUCE-3070 and MAPREDUCE-3363.

        Attachments

        1. MAPREDUCE-3730.patch
          22 kB
          Jason Lowe
        2. MAPREDUCE-3730.patch
          23 kB
          Jason Lowe
        3. MAPREDUCE-3730.patch
          26 kB
          Jason Lowe

          Activity

            People

            • Assignee:
              jlowe Jason Lowe
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: