Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3730

Allow restarted NM to rejoin cluster before RM expires it

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 0.24.0
    • Fix Version/s: 0.23.2
    • Component/s: mrv2, resourcemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry.

      Description

      When a node in the RUNNING state (healthy or unhealthy) is rebooted, the resourcemanager rejects the nodemanager's registration request as a duplicate because it is convinced that the nodemanager is already running on that node. It won't allow that node to rejoin the cluster until the node expiration time elapses which is 10min+ by default. We should allow the NM to rejoin the cluster if it re-registers within the expiration timeout.

      Note that this problem occurs with NMs that are configured to specific ports. If ephemeral ports are used then a NM reboot "works" because the RM thinks the NM registration is for a new node. See the discussions in MAPREDUCE-3070 and MAPREDUCE-3363.

      1. MAPREDUCE-3730.patch
        26 kB
        Jason Lowe
      2. MAPREDUCE-3730.patch
        23 kB
        Jason Lowe
      3. MAPREDUCE-3730.patch
        22 kB
        Jason Lowe

        Activity

          People

          • Assignee:
            Jason Lowe
            Reporter:
            Jason Lowe
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development