[KAFKA-12634] Should checkpoint after restore finished

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 3.5.0
• Component/s: streams
• Labels: new-streams-runtime-should-fix newbie++
• Assignee: Philip Nee
• Reporter: Matthias J. Sax
• Created: 08/Apr/21 16:47
• Updated: 10/Apr/23 22:09
• Resolved: 07/Apr/23 09:21

Description

For state stores, Kafka Streams maintains local checkpoint files to track the offsets of the state store changelog topics. The checkpoint is updated on commit or when a task is closed cleanly.

However, after a successful restore, the checkpoint is not written. Thus, if an instance crashes after restore but before committing, even if the state is on local disk the checkpoint file is missing (indicating that there is no state) and thus state would be restored from scratch.

While for most cases, the time between restore end and next commit is small, there are cases when this time could be large, for example if there is no new input data to be processed (if there is no input data, the commit would be skipped).

Thus, we should write the checkpoint file after a successful restore to close this gap (or course, only for at-least-once processing).

Issue Links

• is related to: KAFKA-13249 Checkpoints do not contain latest offsets on shutdown when using EOS
• links to: GitHub Pull Request #13269