[YARN-4692] [Umbrella] Simplified and first-class support for services in YARN

• Type: New Feature
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Hadoop Flags: Reviewed
• Assignee: Vinod Kumar Vavilapalli
• Reporter: Vinod Kumar Vavilapalli
• Created: 12/Feb/16 21:15
• Updated: 03/Sep/20 04:40

Description

YARN-896 focused on getting the ball rolling on the support for services (long running applications) on YARN.

I’d like propose the next stage of this effort: Simplified and first-class support for services in YARN.

The chief rationale for filing a separate new JIRA is threefold:

• Do a fresh survey of all the things that are already implemented in the project
• Weave a comprehensive story around what we further need and attempt to rally the community around a concrete end-goal, and
• Additionally focus on functionality that YARN-896 and friends left for higher layers to take care of and see how much of that is better integrated into the YARN platform itself.

Issue Links

• incorporates: YARN-4724 [Umbrella] Recognizing services: Special handling of preemption, container reservations etc.
• incorporates: YARN-4725 [Umbrella] Auto-­restart of containers
• incorporates: YARN-4793 [Umbrella] Simplified API layer for services and beyond
• incorporates: YARN-4726 [Umbrella] Allocation reuse for application upgrades
• incorporates: YARN-4758 Enable discovery of AMs by containers
• incorporates: YARN-5025 Container move (relocation) between nodes
• incorporates: YARN-5532 Make ResourceLocalizationService inside NodeManager independent of containers
• incorporates: YARN-4757 [Umbrella] Simplified discovery of services via DNS mechanisms
• incorporates: YARN-5079 [Umbrella] Native YARN framework layer for services and beyond
• is related to: AMBARI-17353 First class support for YARN hosted services
• is related to: YARN-1488 Allow containers to delegate resources to another container
• relates to: YARN-4600 More general services provided to application/container by YARN