[AMBARI-14714] [Umbrella] Multi Everything Architecture

• Type: Epic
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 2.5.0
• Fix Version/s: 3.0.0
• Component/s: ambari-agent, ambari-server, ambari-upgrade, ambari-web, stacks
• Labels: pull-request-available
• Epic Name: Multi Everything Architecture
• Assignee: Jayush Luniya
• Reporter: Jeff Sposetti
• Created: 19/Jan/16 02:55
• Updated: 24/Feb/21 17:09

Description

Multi Stack Services (AMBARI-19621)
Scenario: Deploy HDP & HDF services in same cluster

• Deploy HDFS from HDP and Kafka, Storm, NiFI from HDF in the same cluster.

Scenario: Deploy HDP services from different stack versions in same cluster

• Deploy ZK, HDFS, YARN from HDP-2.5, but deploy latest SPARK from HDP-2.6.

Multiple Service Instances (AMBARI-20463)
Scenario: Multi service instances on same version

• Cluster includes instance of ZooKeeper vX which is being used by HDFS, YARN.
• User wants to add instance of ZooKeeper vX which is being used by STORM and KAFKA

Scenario: Multi service instances on different versions

• Cluster includes instance of SPARK vX.
• User wants to add additional instance of SPARK vY.

Multi Host Component Instances (AMBARI-20465)
Scenario: Multi component instances from a service instance on the same host

• Single host with 128GB RAM, 16 (actual) Cores, 12*4TB disks.
• Single instance of KAFKA broker is unable to utilize all the resources on the host.
• User wants to scale up performance by deploying multiple instances of the KAFKA brokers/host.

Multi Cluster (AMBARI-20466)
Scenario: Manage multiple Hadoop clusters under single Ambari Server

• Customer has multiple small Hadoop clusters and would like to manage and monitor them with a single Ambari Server instance.

Multi Yarn Hosted Services (AMBARI-17353)
Scenario: HBase on YARN

• Deploy second instance of HBase as a long running YARN service.
• Manage YARN hosted service similar to traditional hosted services.
• First class support for Yarn hosted services.

Scenario: Credit Fraud Detection YARN Assembly

• YARN Assembly can have its own ZK, KAFKA etc.
• Manage YARN Assemblies as first-class citizen.

Multi Everything Aware APIs (AMBARI-19620)

• V2 Ambari Rest API to support Multi Everything Architecture

Issue Links

• depends upon: AMBARI-17353 First class support for YARN hosted services
• depends upon: AMBARI-22994 Management Packs: Upgrade Pack Changes
• depends upon: AMBARI-23317 Management Packs: Upgrade Orchestration
• depends upon: AMBARI-19620 Apache Ambari v2 Rest API
• depends upon: AMBARI-19621 Mpack Based Operations Model - Mpack v2
• depends upon: AMBARI-20463 Multi Service Instance Support
• depends upon: AMBARI-20465 Multi Host Component Instance Support
• depends upon: AMBARI-20466 Multi Cluster Support
• depends upon: AMBARI-21152 Config Management in Multi Everything Architecture
• depends upon: AMBARI-22252 Implement Blueprint v2 API changes
• depends upon: AMBARI-12556 Provide ability to apply single patches on top of RU release
• is blocked by: AMBARI-23635 Multi Mpack Aware Advisor
• is blocked by: AMBARI-23009 Refactor Repository/Desired FK for All Tables
• links to: GitHub Pull Request #157
• links to: GitHub Pull Request #289
• links to: GitHub Pull Request #345
• links to: GitHub Pull Request #616
• links to: GitHub Pull Request #791
• links to: GitHub Pull Request #1044
• links to: GitHub Pull Request #1261
• links to: GitHub Pull Request #1464
• links to: GitHub Pull Request #1740
• links to: GitHub Pull Request #1755
• links to: GitHub Pull Request #1761
• links to: GitHub Pull Request #1765
• links to: GitHub Pull Request #2163
• links to: GitHub Pull Request #2215
• links to: GitHub Pull Request #2272
• links to: GitHub Pull Request #2341
• links to: GitHub Pull Request #2400
• links to: Slides