[SLIDER-875] Ability to create an Uber application package with capability to deploy and manage as a single business app

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: Slider 0.70
• Fix Version/s: Slider 0.92
• Component/s: agent, app-package, appmaster, client, core
• Assignee: Billie Rinaldi
• Reporter: Gour Saha
• Created: 14/May/15 23:32
• Updated: 06/Nov/17 22:32
• Resolved: 18/Aug/16 13:51

Description

A business application as we typically refer to, is one that provides value to an end user. Few examples will be, a CRM application, an online advertising application, and a trucking application (to monitor driving habits of truck drivers).

An end user does not understand (or care about) the numerous application components like HBase, Storm, Spark, Kakfa, Tomcat, MySql, Memcached, or Nodejs that are required to build such a business application.

Several such business applications are hosted by cloud vendors like AWS, GCE, Azure, and others. From a cluster management point of view, the IT administrator would benefit from an Uber control of the business application as a whole. The business application owner understands the different components (like Tomcat, Memcached, HBase, etc.) of her/his Uber application. As much as they need fine-grain control of each of these individual applications (which is supported today), they would also benefit from a management control for the Uber app. With Docker becoming popular every day, this will provide a platform to the application owners to define a business application as a conglomeration of Docker containers.

Slider currently is viewed (and used) to package individual applications like HBase, Storm, Kafka, Memcached, and Tomcat. Slider should be able to expose the concept of an Uber application package definition. This Uber definition will be composed of config and resource specifications of the individual application components. Additionally, it will have definitions for Uber management and control, like -

1. Stop, start and flex of the Uber app
2. Dependency specification between the individual applications such that flex of certain components of an application can automatically trigger proportional flex of components in another application
3. Cruise control of the Uber app, on top of what SLIDER-868 will provide for an individual app. Ability to define a skyline for the Uber app, over time and other dimensions.
4. Resource requirements and planning for the Uber app as a whole. Most of the time, an Uber app is functional only when all (or minimum viable) application components are deployed and available. Tomcat running with MySql, Memcached and HBase still waiting for containers, is a useless business application. Slider should be able to do resource calculation and negotiation for the Uber app as a whole. It can work with YARN to get the minimal viable applications of the Uber app running or not bother to run anything (I smell SLAs for vendors and savings for application owners).
5. Ability to define and use multiple YARN labels for the Uber application (in addition to the fine grained label definitions for the individual sub-components of a single app)

I am sure, there are several other benefits which are not identified yet, but this is a start.

Issue Links

• relates to: YARN-5538 Apply SLIDER-875 to yarn-native-services for supporting a composite app-of-apps