[KAFKA-13659] MM2 should read all offset syncs at start up

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.5.0, 3.4.1, 3.3.3
• Component/s: mirrormaker
• Assignee: Greg Harris
• Reporter: Kanalas Vidor
• Created: 09/Feb/22 11:49
• Updated: 27/Nov/23 17:33
• Resolved: 17/Feb/23 22:25

Description

MirrorCheckpointTask uses OffsetSyncStore, and does not check whether OffsetSyncStore managed to read to the "end" of the offset-syncs topic. OffsetSyncStore should fetch the endoffset of the topic at startup, and set a flag when it finally reaches the endoffset in consumption. MirrorCheckpointTask.poll should wait for this flag to be true before doing any in-memory updates and group offset management.

Issue Links

• relates to: KAFKA-15905 Restarts of MirrorCheckpointTask should not permanently interrupt offset translation
• links to: GitHub Pull Request #13178