Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-6465

Add a metrics for the number of records per log

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: log
    • Labels:
      None

      Description

      Currently there are log metrics for:

      • Start offset
      • End offset
      • Size in bytes
      • Number of segments

      I propose to add another metric to track number of record batches in the log. This should provide operators with an idea of how much batching is happening on the producers. Having this metric in one place seems easier than scraping the metric from each producer.

      Having an absolute counter may be infeasible (batches are not assigned sequential IDs), but gauge should be ok. Average batch size can be calculated as (end offset - start offset) / number of batches. This will be heavily skewed for logs with long retention, though.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bobrik Ivan Babrou
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: