[APEXCORE-426] Support work preserving AM recovery

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.6.0
• Labels: apex-hadoop-version
• Assignee: Sandesh Hegde
• Reporter: Thomas Weise
• Created: 10/Apr/16 02:41
• Updated: 05/May/17 10:46
• Resolved: 02/Mar/17 21:46

Description

On app master failure, the streaming containers should continue running.

As of 2.2, YARN will automatically terminate all containers and the replacement app master will relaunch them. Once we move to a newer minimum Hadoop version, we should leverage work preserving restart.

The mechanism in Apex containers to locate the new master process are already in place.

Test Cases:
1. Kill the app-master - only app-master container id should change, all the other containers id should remain same.
2. Kill the app-master and few other containers, make sure that killed containers are recovered.

Issue Links

• is related to: YARN-1490 RM should optionally not kill all containers when an ApplicationMaster exits
• links to: GitHub Pull Request #416