[KAFKA-2569] Kafka should write its metrics to a Kafka topic

• Type: New Feature
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: metrics
• Assignee: Unassigned
• Reporter: James Cheng
• Created: 22/Sep/15 20:20
• Updated: 03/Aug/18 00:27

Description

Kafka is often used to hold and transport monitoring data.

In order to monitor Kafka itself, Kafka currently exposes many metrics via JMX, which require using a tool to pull the JMX metrics, and then write them to the monitoring system.

It would be convenient if Kafka could simply send its metrics to a Kafka topic. This would make most sense if the Kafka topic was in a different Kafka cluster, but could still be useful even if it was sent to a topic in the same Kafka cluster.

Of course, if sent to the same cluster, it would not be accessible if the cluster itself was down.

This would allow monitoring of Kafka itself without requiring people to set up their own JMX-to-monitoring-system pipelines.