Details
Description
[See overview doc at YARN-4692, modifying and copy-pasting some of the relevant pieces and sub-section 3.3.2 to track the specific sub-item.]
Bringing a new service on YARN today is not a simple experience. The APIs of existing frameworks are either too low level (native YARN), require writing new code (for frameworks with programmatic APIs ) or writing a complex spec (for declarative frameworks).
In addition to building critical building blocks inside YARN (as part of other efforts at YARN-4692), we should also look to simplifying the user facing story for building services. Experience of projects like Slider building real-life services like HBase, Storm, Accumulo, Solr etc gives us some very good learnings on how simplified APIs for building services will look like.
To this end, we should look at a new simple-services API layer backed by REST interfaces. The REST layer can act as a single point of entry for creation and lifecycle management of YARN services. Services here can range from simple single-component apps to the most complex, multi-component applications needing special orchestration needs.
We should also look at making this a unified REST based entry point for other important features like resource-profile management (YARN-3926), package-definitions' lifecycle-management and service-discovery (YARN-913 / YARN-4757). We also need to flesh out its relation to our present much lower level REST APIs (YARN-1695) in YARN for application-submission and management.
Attachments
Attachments
Issue Links
- is part of
-
YARN-4692 [Umbrella] Simplified and first-class support for services in YARN
-
- Reopened
-
- is related to
-
AMBARI-17353 First class support for YARN hosted services
-
- Open
-