[KAFKA-16622] Mirromaker2 first Checkpoint not emitted until consumer group fully catches up once

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.7.0, 3.6.2, 3.8.0
• Fix Version/s: 3.8.0, 3.7.1
• Component/s: mirrormaker
• Assignee: Edoardo Comar
• Reporter: Edoardo Comar
• Created: 25/Apr/24 15:22
• Updated: 05/Jun/24 00:40
• Resolved: 24/May/24 20:30

Description

We observed an excessively delayed emission of the MM2 Checkpoint record.
It only gets created when the source consumer reaches the end of a topic. This does not seem reasonable.

In a very simple setup :

Tested with a standalone single process MirrorMaker2 mirroring between two single-node kafka clusters(mirromaker config attached) with quick refresh intervals (eg 5 sec) and a small offset.lag.max (eg 10)

create a single topic in the source cluster
produce data to it (e.g. 10000 records)
start a slow consumer - e.g. fetching 50records/poll and pausing 1 sec between polls which commits after each poll

watch the Checkpoint topic in the target cluster

bin/kafka-console-consumer.sh --bootstrap-server localhost:9192 \
--topic source.checkpoints.internal \
--formatter org.apache.kafka.connect.mirror.formatters.CheckpointFormatter \
--from-beginning

-> no record appears in the checkpoint topic until the consumer reaches the end of the topic (ie its consumer group lag gets down to 0).