[MAPREDUCE-5567] [Umbrella] Stabilize MR framework w.r.t ResourceManager restart

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Assignee: Unassigned
• Reporter: Vinod Kumar Vavilapalli
• Created: 05/Oct/13 22:32
• Updated: 08/May/15 18:08
• Resolved: 08/May/15 18:08

Description

There are a bunch of tickets tracking MR AM's issues w.r.t RM restart. Consolidating them here so that we don't make contradictory fixes accross JIRAs.

Issue Links

• is related to: YARN-128 [Umbrella] RM Restart Phase 1: State storage and non-work-preserving recovery
• is related to: YARN-556 [Umbrella] RM Restart phase 2 - Work preserving restart

Sub-Tasks

• MR AM should exit when unregister() throws exception: Closed
• Clients should be notified job finished only after job successfully unregistered: Closed
• MRAppMaster#shutDownJob shouldn't send job end notification before checking isLastRetry: Closed
• Job history should not be flushed to JHS until AM gets unregistered: Open
• Job can fail when RM restarts after staging dir is cleaned but before MR successfully unregister with RM: Closed
• Succeed job tries to restart after RMrestart: Resolved
• Historyserver does not refresh the result of restarted jobs after RM restart: Closed
• JobClient exit whenever RM issue Reboot command to 1st attempt App Master.: Closed
• MR job succeeds and exits even when unregister with RM fails: Resolved
• MR app master deletes staging dir when sent a reboot command from the RM: Closed
• Backport MAPREDUCE-5086 - MR app master deletes staging dir when sent a reboot command from the RM: Resolved