[YARN-1210] During RM restart, RM should start a new attempt only when previous attempt exits for real

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.3.0
• Hadoop Flags: Reviewed
• Assignee: Omkar Vinit Joshi
• Reporter: Vinod Kumar Vavilapalli
• Created: 17/Sep/13 21:07
• Updated: 17/Jun/14 21:43
• Resolved: 19/Nov/13 05:18

Description

When RM recovers, it can wait for existing AMs to contact RM back and then kill them forcefully before even starting a new AM. Worst case, RM will start a new AppAttempt after waiting for 10 mins ( the expiry interval). This way we'll minimize multiple AMs racing with each other. This can help issues with downstream components like Pig, Hive and Oozie during RM restart.

In the mean while, new apps will proceed as usual as existing apps wait for recovery.

This can continue to be useful after work-preserving restart, so that AMs which can properly sync back up with RM can continue to run and those that don't are guaranteed to be killed before starting a new attempt.

Issue Links

• duplicates: YARN-1373 Transition RMApp and RMAppAttempt state to RUNNING after restart for recovered running apps
• is blocked by: YARN-891 Store completed application information in RM state store