[KAFKA-15079] Offset expirations in topics with less event generation results in messages that take months to be delivered.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: group-coordinator
• Assignee: Ritika Reddy
• Reporter: Ritika Reddy
• Reviewer: David Jacot
• Created: 09/Jun/23 23:30
• Updated: 10/Jun/23 16:28

Description

It was observed that messages were taking months to be delivered in certain pipelines. The offsets had been reset and consumption restarted from messages that were a couple months old.

On further investigation the team found that once the offsets.retention.minutes (7 days) elapses and there's no subscribers to the topic, the offsets are subject to immediate expiration. This can be a problem when we close the current consumer and the retention period has already passed due to the infrequent nature of offset commits. The offsets then expire before a new consumer can come up and subscribe to the topic again. Since only one consumer could be subscribed to the topic it will be removed from the subscription list as soon as the consumer leaves. 

Two things to keep in mind here are :-

1. The consumer group has different consumers subscribed to different topics.

1. Events aren't generated very frequently to these topics.

https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/coordinator/group/GroupMetadata.scala#LL766C1-L825C4