[MAPREDUCE-3730] Allow restarted NM to rejoin cluster before RM expires it

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.23.1, 2.0.0-alpha
• Fix Version/s: 0.23.2
• Component/s: mrv2, resourcemanager
• Target Version/s: 0.23.2
• Hadoop Flags: Reviewed
• Release Note: Modified RM to allow restarted NMs to be able to join the cluster without waiting for expiry.
• Assignee: Jason Darrell Lowe
• Reporter: Jason Darrell Lowe
• Created: 26/Jan/12 00:12
• Updated: 10/Mar/15 04:32
• Resolved: 24/Feb/12 21:43

Description

When a node in the RUNNING state (healthy or unhealthy) is rebooted, the resourcemanager rejects the nodemanager's registration request as a duplicate because it is convinced that the nodemanager is already running on that node. It won't allow that node to rejoin the cluster until the node expiration time elapses which is 10min+ by default. We should allow the NM to rejoin the cluster if it re-registers within the expiration timeout.

Note that this problem occurs with NMs that are configured to specific ports. If ephemeral ports are used then a NM reboot "works" because the RM thinks the NM registration is for a new node. See the discussions in MAPREDUCE-3070 and MAPREDUCE-3363.