[YARN-1360] Rework Distributed Shell to be a better model of how people should write YARN applications

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.2.0
• Component/s: applications/distributed-shell
• Assignee: Unassigned
• Reporter: Steve Loughran
• Created: 29/Oct/13 15:00
• Updated: 29/Oct/13 15:03

Description

Distributed Shell works as an example, but it's not the right architecture for something you'd want in production -instead its design runs a risk of setting a bad example for others to follow.

Instead it should

• be decomposed into a set of services each with their own responsibilities
• split the 'model' of its cluster into its own classes, a model that can then be unit tested outside of the AM
• factor out all container launching into its own service -and use a thread pool to avoid scalability limits
• provide a demonstration (classic) RPC service to show how to implement this
• tests