[AMBARI-25301] Ambari metrics HA does not work in 2.7.x for Hbase/any /services

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.7.3
• Fix Version/s: 2.8.0
• Component/s: ambari-metrics
• Labels: pull-request-available
• Assignee: amarnath reddy pappu
• Reporter: amarnath reddy pappu
• Created: 11/Jun/19 19:58
• Updated: 25/Oct/22 16:13
• Resolved: 20/Jun/19 17:48

Description

Steps to reproduce:
-------------------------
1. Install Ambari 2.7.3 and HDP3.1 cluster.
2. Add Ambari metrics service.
3. From host page, add another ambari metrics collector.
4. Hbase would not be able to connect to collector.

Root cause:
hadop-sink is not able to spit these below hosts and end of with bad URL.

hbase.sink.timeline.collector.hosts=c2111-node3.hdp.com,c2111-node2.hdp.com

2019-06-11 19:46:02,731 DEBUG [timeline] timeline.HadoopTimelineMetricsSink: Trying to find live collector host from : [c2111-node3.hdp.com,c2111-node2.hdp.com]
2019-06-11 19:46:02,731 DEBUG [timeline] timeline.HadoopTimelineMetricsSink: Requesting live collector nodes : http://c2111-node3.hdp.com,c2111-node2.hdp.com:6188/ws/v1/timeline/metrics/livenodes
2019-06-11 19:46:02,732 DEBUG [timeline] timeline.HadoopTimelineMetricsSink: Unable to connect to collector, http://c2111-node3.hdp.com,c2111-node2.hdp.com:6188/ws/v1/timeline/metrics/livenodes

In HadoopTimelineMetricsSink.java

    collectorHosts = parseHostsStringArrayIntoCollection(conf.getStringArray(COLLECTOR_HOSTS_PROPERTY));

conf.getStringArray(COLLECTOR_HOSTS_PROPERTY) is not actually returning the String array.

Issue Links

• duplicates: AMBARI-25326 AMS - no HBase and Hive metrics post-upgrade when using 2 collectors
• links to: GitHub Pull Request #45
• links to: GitHub Pull Request #3006