Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4179

Replication Throttling: Add Usability Metrics PartitionBytesInRate & SumReplicaLag

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.10.1.0
    • Fix Version/s: None
    • Component/s: replication
    • Labels:
      None

      Description

      Add two new metrics to Kafka

      PartitionBytesInRate: Equivalent to BytesInPerSec, but at a partition level (i.e. total traffic - throttled and not throttled). This is required for estimating how long a rebalance will take to complete. B/s. See usability section below.

      SumReplicaLag: This is the sum of all replica lag values on the broker. This metric is used to monitor progress of a rebalance and is particularly useful for determining if the rebalance has become stuck due to an overly harsh throttle value (as the metric will stop decreasing).

      As covered in KIP-73 https://cwiki.apache.org/confluence/display/KAFKA/KIP-73+Replication+Quotas

      These make it possible for an administrator to calculate how long a rebalance will take.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              benstopford Ben Stopford
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: