[AMBARI-20562] Perf: Asynchronous event handling server/agent

• Type: Epic
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.7.0
• Component/s: ambari-agent, ambari-server
• Labels: performance
• Epic Name: Asynchronous event handling server/agent
• Assignee: Siddharth Wagle
• Reporter: Siddharth Wagle
• Created: 24/Mar/17 17:05
• Updated: 04/Dec/18 05:05
• Resolved: 04/Dec/18 05:04

Description

Ambari's internal event model is heavily fragmented with some of the events that result in state transitions being synchronous in nature. The proposal of this Epic is to standardize event handling on the server-side and utilize its asynchronous nature to make architecture changes that are geared to towards scaling up the performance of Ambari server. The goal is to support 5K active nodes with this new architecture.

To summarize the entailment:

• Write a Global Event Observer to allow clients to subscribe to server side events
• Use websocket protocol to reduce network chatter between agent and server
• Reduce the work server has to do to generate STATUS commands for heartbeats.
• Make execution commands leaner replacing the need for providing topology info and configs with every command, instead rely on event model and agent in-memory state.
• Re-architect the agent to make it more intelligent towards maintaining topology info, daemon status and configs.
• Reduce computation requirements for every heartbeat and allow for parallel processing of agent side events