Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3730

Allow restarted NM to rejoin cluster before RM expires it

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 0.23.1, 2.0.0-alpha
    • 0.23.2
    • mrv2, resourcemanager
    • None
    • Reviewed
    • Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry.

    Description

      When a node in the RUNNING state (healthy or unhealthy) is rebooted, the resourcemanager rejects the nodemanager's registration request as a duplicate because it is convinced that the nodemanager is already running on that node. It won't allow that node to rejoin the cluster until the node expiration time elapses which is 10min+ by default. We should allow the NM to rejoin the cluster if it re-registers within the expiration timeout.

      Note that this problem occurs with NMs that are configured to specific ports. If ephemeral ports are used then a NM reboot "works" because the RM thinks the NM registration is for a new node. See the discussions in MAPREDUCE-3070 and MAPREDUCE-3363.

      Attachments

        1. MAPREDUCE-3730.patch
          26 kB
          Jason Darrell Lowe
        2. MAPREDUCE-3730.patch
          23 kB
          Jason Darrell Lowe
        3. MAPREDUCE-3730.patch
          22 kB
          Jason Darrell Lowe

        Activity

          People

            jlowe Jason Darrell Lowe
            jlowe Jason Darrell Lowe
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: