[KAFKA-8833] Metric for Broker Consumer Lag in size and time

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: metrics
• Assignee: Unassigned
• Reporter: NIKHIL
• Created: 26/Aug/19 15:20
• Updated: 23/Oct/19 05:22

Description

In many use cases consumers are expected to fetch at the tail end of the log. The proposal is to have stats on consumer lag in terms for bytes as well as time. We will add broker level metrics aggregated across all partitions. Both metrics will be histogram.
 

• FetchLagTimeInMs: Histogram that will measure fetch log lag using the timestamp of the messages being fetched  
• FetchLagBytes: Histogram that will measure the fetch log lag by calculating the byte lag of the message being fetched
   

Issue Links

• links to: GitHub Pull Request #7273