[YARN-6127] Add support for work preserving NM restart when AMRMProxy is enabled

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Component/s: amrmproxy, nodemanager
• Target Version/s: 2.9.0, 3.0.0-beta1
• Hadoop Flags: Incompatible change

Release Note

This breaks rolling upgrades because it changes the major version of the NM state store schema. Therefore when a new NM comes up on an old state store it crashes.

The state store versions for this change have been updated in YARN-6798.

• Assignee: Botong Huang
• Reporter: Subramaniam Krishnan
• Created: 27/Jan/17 19:58
• Updated: 02/Apr/18 23:46
• Resolved: 22/Jun/17 22:10

Description

YARN-1336 added the ability to restart NM without loosing any running containers. In a Federated YARN environment, there's additional state in the AMRMProxy to allow for spanning across multiple sub-clusters, so we need to enhance AMRMProxy to support work-preserving restart.

Issue Links

• blocks: YARN-6704 Add support for work preserving NM restart when FederationInterceptor is enabled in AMRMProxyService
• is related to: YARN-1336 [Umbrella] Work-preserving nodemanager restart
• relates to: YARN-8110 AMRMProxy recover should catch for all throwable to avoid premature exit
• relates to: YARN-6798 Fix NM startup failure with old state store due to version mismatch