[KAFKA-13562] Mirror Maker 2 Negative Offsets

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.8.1
• Fix Version/s: 3.5.0, 3.4.1, 3.3.3
• Component/s: connect, mirrormaker
• External issue URL: https://github.com/strimzi/strimzi-kafka-operator/discussions/6082
• Assignee: Unassigned
• Reporter: David Urton
• Created: 21/Dec/21 20:38
• Updated: 02/Aug/23 16:52
• Resolved: 02/Aug/23 16:52

Description

We recently started a migration from a traditional self-managed Apache Kafka cluster to Strimzi. We enabled group offset sync and something I'm struggling to fully understand is why the mirrored group has a current-offset matching (or close to matching) the source offset, but then the log-end-offset essentially starts over resulting in a negative lag. Is this just aesthetic? The lag is slowly moving closer to 0 but some of our negative lags are very "large". Several hundred million. I verified all the data has moved over, but I'm struggling on this lag issue.

Any help clearing this up is much appreciated!

Source cluster is 2.4.1 destination is 2.8.1. Detailed info in the Github URL.

Issue Links

• relates to: KAFKA-12468 Initial offsets are copied from source to target cluster
• Testing discovered: KAFKA-14663 High throughput topics can starve low-throughput MM2 offset syncs