Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-24295

Integrate OpenTSDB + replace Ambari Metrics Collector

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.5.2
    • Fix Version/s: None
    • Component/s: ambari-metrics
    • Labels:
      None
    • Environment:

      HDP 2.6

      Description

      Request to integrate OpenTSDB for metrics collection and replace the Ambari Metrics Collector with OpenTSDB.

      1. OpenTSDB is a much more widely used, tested and scaled metrics time series database
      2. OpenTSDB is already run by Hortonworks customers for doing real world metrics collection at scale. For example, on one same cluster where AMS is collecting under 2500 metrics, OpenTSDB is collecting over 250,000 different metrics with much higher dimensionality of tags (AMS monitors a handful of hosts while OpenTSDB monitors nearly all hosts in company) on the same hardware
      3. HBase - both AMS and OpenTSDB already run on HBase, so only AMS Collector needs to be swapped out and metrics monitors updated (metrics monitors might be better replaced by Telegraf in the interests of standardization)
      4. Grafana Compatibility - standard centralised Grafana as found in companies doesn't have the data source to query Ambari AMS but can query OpenTSDB natively, so OpenTSDB is more compatible with the outside monitoring ecosystem
      5. Scaling - Ambari should be able to scale OpenTSDB easily from out of the box smaller deployment just to support local cluster, to any scale for entire company solution just by adding RegionServers and TSD instances, further driving Hadoop + HBase adoption and revenue. While AMS can benefit from HBase scaling, I don't think AMS is designed for scale and general purpose like OpenTSDB - there isn't an option for multiple load balancer Metrics Collectors like with OpenTSDB's TSDs
      6. MapR has already been bundling OpenTSDB for several releases now for their metrics collection

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              harisekhon Hari Sekhon
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: