Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8833

Metric for Broker Consumer Lag in size and time

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: metrics
    • Labels:
      None

      Description

      In many use cases consumers are expected to fetch at the tail end of the log. The proposal is to have stats on consumer lag in terms for bytes as well as time. We will add broker level metrics aggregated across all partitions. Both metrics will be histogram.
       

      • FetchLagTimeInMs: Histogram that will measure fetch log lag using the timestamp of the messages being fetched  
      • FetchLagBytes: Histogram that will measure the fetch log lag by calculating the byte lag of the message being fetched
         

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                rite2nikhil NIKHIL
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated: