[NIFI-4809] Implement a SiteToSiteMetricsReportingTask

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Component/s: Extensions
• Assignee: Pierre Villard
• Reporter: Pierre Villard
• Created: 23/Jan/18 22:12
• Updated: 25/Apr/18 17:51
• Resolved: 11/Apr/18 19:02

Description

At the moment there is an AmbariReportingTask to send the NiFi-related metrics of the host to the Ambari Metrics Service. In a multi-cluster configuration, or when working with MiNiFi (Java) agents, it might not be possible for all the NiFi instances (NiFi and/or MiNiFi) to access the AMS REST API.

To solve this problem, a solution would be to implement a SiteToSiteMetricsReportingTask to send the data via S2S to the "main" NiFi instance/cluster that will be able to publish the metrics into AMS (using InvokeHTTP). This way, it is possible to have the metrics of all the instances exposed in one AMS instance.

I propose to send the data formatted as we are doing right now in the Ambari reporting task. If needed, it can be easily converted into another schema using the record processors once received via S2S.

Issue Links

• relates to: NIFI-4996 Duplicate copies of record reader/writer controller services shown
• relates to: NIFI-5122 Add record writer to S2S Reporting Tasks
• links to: GitHub Pull Request #2430
• links to: GitHub Pull Request #2575