[AMBARI-5707] Replace Ganglia with high performance and pluggable Metrics System

• Type: Epic
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.6.0
• Fix Version/s: 2.0.0
• Component/s: ambari-agent, ambari-server
• Epic Name: Ambari Metrics System
• Assignee: Siddharth Wagle
• Reporter: Siddharth Wagle
• Created: 07/May/14 23:52
• Updated: 12/May/16 02:57
• Resolved: 17/Mar/15 19:17

Description

Ambari Metrics System

• Ability to collect metrics from Hadoop and other Stack services
• Ability to retain metrics at a high precision for a configurable time period (say 5 days)
• Ability to automatically purge metrics after retention period
• At collection time, provide clear integration point for external system (such as TSDB)
• At purge time, provide clear integration point for metrics retention by external system
• Should provide default options for external metrics retention (say “HDFS”)
• Provide tools / utilities for analyzing metrics in retention system (say “Hive schema, Pig scripts, etc” that can be used with the default retention store “HDFS”)

System Requirements

• Must be portable and platform independent
• Must not conflict with any existing metrics system (such as Ganglia)
• Must not conflict with existing SNMP infra
• Must not run as root
• Must have HA story (no SPOF)

Usage

• Ability to obtain metrics from Ambari REST API (point in time and temporal)
• Ability to view metric graphs in Ambari Web (currently, fixed)
• Ability to configure custom metric graphs in Ambari Web (currently, we have metric graphs “fixed” into the UI)
• Need to improve metric graph “navigation” in Ambari Web (currently, metric graphs do not allow navigation at arbitrary timeframes, but only at ganglia aggregation intervals)
• Ability to “view cluster” at point in time (i.e. see all metrics at that point)
• Ability to define metrics (and how + where to obtain) in Stack Definitions

Issue Links

• supercedes: AMBARI-15901 Ambari Metrics System Distributed mode - multiple Collectors
• supercedes: AMBARI-10144 Ambari Metrics System v0.2 - Enhancements
• links to: Review board - Trunk merge