[AMBARI-25278] Fix aggregate metric in Kafka's Grafana dashboard

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.3
• Fix Version/s: 2.7.4
• Component/s: ambari-server
• Labels: pull-request-available
• Assignee: Balázs Bence Sári
• Reporter: Balázs Bence Sári
• Created: 13/May/19 18:46
• Updated: 27/Jun/19 15:10
• Resolved: 27/Jun/19 15:10

Description

The following metrics display aggregate numbers over time on Grafana (they are ever increasing). They should show timely rate instead.

The following metrics are affected:

Kafka Home:

kafka.server.BrokerTopicMetrics.BytesInPerSec.count
kafka.server.BrokerTopicMetrics.BytesOutPerSec.count
kafka.server.BrokerTopicMetrics.MessagesInPerSec.count
kafka.network.RequestMetrics.RequestsPerSec.request.Produce.count
kafka.network.RequestMetrics.RequestsPerSec.request.FetchConsumer.count
kafka.controller.ControllerStats.LeaderElectionRateAndTimeMs.count
kafka.controller.ControllerStats.UncleanLeaderElectionsPerSec.count
kafka.server.ReplicaManager.IsrShrinksPerSec.count
kafka.server.ReplicaManager.IsrExpandsPerSec.count

Kafka Hosts:

kafka.server.BrokerTopicMetrics.BytesInPerSec.count
kafka.server.BrokerTopicMetrics.BytesOutPerSec.count
kafka.server.BrokerTopicMetrics.MessagesInPerSec.count
kafka.network.RequestMetrics.RequestsPerSec.request.Produce.count
kafka.network.RequestMetrics.RequestsPerSec.request.FetchConsumer.count
kafka.server.ReplicaManager.IsrShrinksPerSec.count
kafka.server.ReplicaManager.IsrExpandsPerSec.count

Kafka Topics

kafka.server.BrokerTopicMetrics.BytesInPerSec.topic.*.count
kafka.server.BrokerTopicMetrics.BytesOutPerSec.topic.*.count
kafka.server.BrokerTopicMetrics.BytesInPerSec.topic.*.count
kafka.server.BrokerTopicMetrics.MessagesInPerSec.topic.*.count
kafka.server.BrokerTopicMetrics.TotalProduceRequestsPerSec.topic.*.count

They are typically needed to be converted from something like this:

  "targets": [
    {
      "aggregator": "avg",
      "alias": "ISR Shrinks",
      "app": "kafka_broker",
      "downsampleAggregator": "avg",
      "errors": {},
      "metric": "kafka.server.ReplicaManager.IsrShrinksPerSec.count",
      "precision": "default",
      "refId": "A",
      "transform": "none",
      "transformData": "none",
      "seriesAggregator": "none"
    }
  ],

to this:

  "targets": [
    {
      "aggregator": "avg",
      "alias": "ISR Shrinks",
      "app": "kafka_broker",
      "downsampleAggregator": "avg",
      "errors": {},
      "metric": "kafka.server.ReplicaManager.IsrShrinksPerSec.count",
      "precision": "default",
      "refId": "A",
      "transform": "rate",
      "transformData": "none",
      "seriesAggregator": "none"
    }
  ],

with topics metrics, aggregator must be changed to none in addition.

The Grafana dashboards are stored in the stack definitions.

Issue Links

• links to: GitHub Pull Request #2982