[KAFKA-13936] Invalid consumer lag when monitoring from a kafka streams application

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.3.0
• Component/s: streams
• Assignee: Prashanth Joseph Babu
• Reporter: Prashanth Joseph Babu
• Created: 25/May/22 07:56
• Updated: 30/Jun/22 16:32
• Resolved: 30/Jun/22 16:31

Description

I have a kafka streams application and I'm trying to monitor the consumer lag via stream metrics.

Here's some code snippet

metrics = streams.metrics();
            lag = 0;
            for (Metric m : metrics.values()) {
                tags = m.metricName().tags();
                if ( m.metricName().name().equals(MONITOR_CONSUMER_LAG) && tags.containsKey(MONTOR_TAG_TOPIC) && 
                    tags.get(MONTOR_TAG_TOPIC).equals(inputTopic) ) {
                    partitionLag = Float.valueOf(m.metricValue().toString()).floatValue();
                    if ( !partitionLag.isNaN() ) {
                        lag += partitionLag;
                    }
                }
            }

Here MONITOR_CONSUMER_LAG is records-lag-max.

However these numbers dont match with the consumer lag we see in the kafka UI . is records-lag-max the right metric to track for a kafka streams application when the objective is to get consumer lag?