[YARN-556] [Umbrella] RM Restart phase 2 - Work preserving restart

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: graceful, resourcemanager, rolling upgrade
• Target Version/s: 2.6.0
• Assignee: Unassigned
• Reporter: Bikas Saha
• Created: 08/Apr/13 20:49
• Updated: 27/Jan/17 20:00
• Resolved: 03/May/15 01:03

Description

YARN-128 covered storing the state needed for the RM to recover critical information. This umbrella jira will track changes needed to recover the running state of the cluster so that work can be preserved across RM restarts.

Issue Links

• is depended upon by: YARN-666 [Umbrella] Support rolling upgrades in YARN
• is related to: YARN-128 [Umbrella] RM Restart Phase 1: State storage and non-work-preserving recovery
• relates to: YARN-6128 Add support for AMRMProxy HA
• relates to: MAPREDUCE-5567 [Umbrella] Stabilize MR framework w.r.t ResourceManager restart
• relates to: YARN-149 [Umbrella] ResourceManager (RM) Fail-over

Sub-Tasks

• ApplicationMasterService to allow Register of an app that was running before restart: Closed
• AM should implement Resync with the ApplicationMasterService instead of shutting down: Closed
• After restart NM should resync with the RM without killing containers: Closed
• Common work to re-populate containers’ state into scheduler: Closed
• Capacity scheduler to re-populate container allocation state: Resolved
• Fair scheduler to re-populate container allocation state: Closed
• FIFO scheduler to re-populate container allocation state: Resolved
• Ensure all completed containers are reported to the AMs across RM restart: Closed
• Transition RMApp and RMAppAttempt state to RUNNING after restart for recovered running apps: Resolved
• Revisit AM link being broken for work preserving restart: Resolved
• Recover Unmanaged AMs: Resolved
• Mark Idempotent/AtMostOnce annotations to ApplicationMasterProtocol for RM fail over: Closed
• Fix ordering of starting services inside the RM: Resolved
• Threshold for RM to accept requests from AM after failover: Closed
• Merge some of the common lib code in schedulers: Closed
• ContainerId creation after work preserving restart is broken: Closed
• Replace RegisterNodeManagerRequest#ContainerStatus with a new NMContainerStatus: Closed
• Recover missing container information: Closed
• Ensure distributed shell work with RM work-preserving recovery: Closed
• Update ContainerId#toString() to avoid conflicts before and after RM restart: Closed
• ContainerId can overflow with RM restart: Closed
• AM release request may be lost on RM restart: Closed
• Add containers to launchedContainers list in RMNode on container recovery: Closed
• Marking ContainerId#getId as deprecated: Closed
• RM should not recover containers from previously failed attempt when AM restart is not enabled: Closed
• Possible livelock in CapacityScheduler when RM is recovering apps: Closed
• Update ConverterUtils#toContainerId to parse epoch: Closed
• Updating ContainerTokenIdentifier#read/write to use ContainerId#getContainerId: Closed
• Add a percentage-node threshold for RM to wait for new allocations after restart/failover: Open
• Distributed shell AM may re-launch containers if RM work preserving restart happens: Patch Available
• TestWorkPreservingRMRestart: Augment FS tests with queue and headroom checks: Closed
• NPE when RM tries to transfer state from previous attempt on recovery: Resolved
• Document work-preserving RM restart: Closed
• Make work-preserving-recovery the default mechanism for RM recovery: Closed