[YARN-420] Enable the RM to work with AM's that are not managed by it

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha1
• Fix Version/s: 2.0.2-alpha
• Component/s: applications/unmanaged-AM-launcher
• Labels: incom mrv2
• Assignee: Bikas Saha
• Reporter: Bikas Saha
• Created: 10/Jul/12 20:37
• Updated: 12/May/16 18:30
• Resolved: 15/Jul/12 21:48

Description

Currently, the RM itself manages the AM by allocating a container for it and negotiating the launch on the NodeManager and manages the AM lifecycle. Thereafter, the AM negotiates resources with the RM and launches tasks to do the real work.
It would be a useful improvement to enhance this model by allowing the AM to be launched independently by the client without requiring the RM. These AM's would be launched on a gateway machine that can talk to the cluster. This would open up new use cases such as the following
1) Easy debugging of AM, specially during initial development. Having the AM launched on an arbitrary cluster node makes it hard to looks at logs or attach a debugger to the AM. If it can be launched locally then these tasks would be easier.
2) Running AM's that need special privileges that may not be available on machines managed by the NodeManager

Issue Links

• blocks: YARN-419 Add client side for Unmanaged-AMs
• relates to: YARN-255 Support secure AM launch for unmanaged AM's