[KAFKA-13732] Improve offsets.retention.minutes document to specify it's consumer per topic instead whole consumer group

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: documentation
• Assignee: Unassigned
• Reporter: Yujie Li
• Created: 14/Mar/22 06:20
• Updated: 14/Mar/22 06:28

Description

Hey,

I recently realize consumer group offset retention behavior could improve in kafka document. https://kafka.apache.org/documentation/#brokerconfigs_offsets.retention.minutes

The scenario I have is:

1. A consumer group[ConsumerGroupA] with multiple consumers subscribe multiple topics [TopicA, TopicB]
2. Remove Topic B from ConsumerGroupA, but the producer still publish message to Topic B, ConsumerGroupA still subscribe Topic A
3. After `offsets.retention.minutes` reached, ConsumerGroupA's offset for topic B is gone.
4. Re-add TopicB to ConsumerGroupA, offset reset to earliest / lastest

 

From my understand this configuration more topic oriented than consumer group describe in the doc and I wish it could be more specific.

 

And here is my proposing doc change:

After a consumer group loses all its consumers (i.e. becomes empty) with a topic, its offsets will be kept for this retention period before getting discarded. For standalone consumers (using manual assignment), offsets will be expired after the time of last commit plus this retention period.

 

Besides, I wasn't find the place I can patch this small change in apache/kafka repo. If anyone can point me the place and agree this document update make sense, I can create a path for it.

 

Let me know what do you think!

 

Thanks,

Yujie