[KAFKA-7942] Monitoring consumer lag by timestamp

• Type: Wish
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: consumer, metrics
• Assignee: Unassigned
• Reporter: Yining Liu
• Created: 18/Feb/19 10:14
• Updated: 21/Feb/19 08:26

Description

Currently, we can collect num of messages consumer lagging behind producer. However, if messages come into a partition sporadically, we are not easy to define consumer lag alert threshold. If we know how long consumer lagging behind latest message, it must be helpful.

There are two options:

1. expose per partition lag time by consumer jmx metrics
2. expose consumer lag time in kafka-consumer-groups.sh