[KAFKA-12261] Splitting partition causes message loss for consumers with auto.offset.reset=latest

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 3.6.0
• Assignee: Haruki Okada
• Reporter: Haruki Okada
• Created: 31/Jan/21 11:44
• Updated: 29/Feb/24 13:38
• Resolved: 14/Jul/23 03:14

Description

As of now, auto.offset.reset of ConsumerConfig is "latest" by default.

 

This could be a pitfall that causes message delivery loss when we split topic's partitions like below:

Say we have a topic-X which have only 1 partition.

1. split topic-X to 2 partitions by kafka-topics.sh --alter --topic topic-X --partitions 2 (topic-X-1 is added)
2. producer knows that new partitions are added by refreshing metadata. starts to produce to topic-X-1
3. bit later, consumer knows that new partitions are added and triggering consumer rebalance, then starts consuming topic-X-1

• • upon starting consumption, it resets its offset to log-end-offset

If the producer sent several records before 3, they could be not-delivered to the consumer.

 

 

This behavior isn't preferable in most cases, so it should be documented in AUTO_OFFSET_RESET_DOC at least.

Issue Links

• duplicates: KAFKA-12478 Consumer group may lose data for newly expanded partitions when add partitions for topic if the group is set to consume from the latest
• is duplicated by: KAFKA-15185 Consumers using the latest strategy may lose data after the topic adds partitions
• links to: GitHub Pull Request #10167