[KAFKA-15625] Do not flush global state store at each commit

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.8.0
• Component/s: streams
• Assignee: Ayoub Omari
• Reporter: Bruno Cadonna
• Created: 17/Oct/23 14:53
• Updated: 05/Mar/24 08:54
• Resolved: 05/Mar/24 08:38

Description

Global state stores are flushed at each commit. While that is not a big issue with at-least-once processing mode since the commit interval is by default 30s, it might become an issue with EOS where the commit interval is 200ms by default.
One option would be to flush and checkpoint global state stores when the delta of the content exceeds a given threshold as we do for other stores. See https://github.com/apache/kafka/blob/a1f3c6d16061566a4f53c72a95e2679b8ee229e0/streams/src/main/java/org/apache/kafka/streams/processor/internals/AbstractTask.java#L97

Issue Links

• links to: GitHub Pull Request #15361