[KAFKA-14294] Kafka Streams should commit transaction when no records are processed

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.2.1
• Fix Version/s: 3.4.0
• Component/s: streams
• Assignee: A. Sophie Blee-Goldman
• Reporter: Vicky Papavasileiou
• Created: 12/Oct/22 16:18
• Updated: 30/Aug/23 21:37
• Resolved: 06/Dec/22 00:33

Description

Currently, if there are no records to process in the input topic, a transaction does not commit. If a custom punctuator code is writing to a state store (which is common practice) the producer gets fenced when trying to write to the changelog topic. This throws a TaskMigratedException and causes a rebalance. 

A better approach would be to commit a transaction even when there are no records processed as to allow the punctuator to make progress. 

Issue Links

• links to: GitHub Pull Request #12835