[KAFKA-6838] Transaction timeout after punctuation results in ProducerFencedException

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.1.0
• Component/s: streams
• Assignee: Unassigned
• Reporter: Felix Dsouza
• Created: 30/Apr/18 09:32
• Updated: 16/May/18 20:33
• Resolved: 16/May/18 20:33

Description

It looks like the EOS/transaction handling for stream punctuation does not work correctly.

We have encountered ProducerFencedException in a specific case where new events do not arrive before the transaction started by punctuation times out.

Following squence of events trigger this scenario.

 1. Punctuation is triggered, we delete some records from the State store (logging enabled) . A transaction is registered on the broker. txnState=Ongoing
2. There are no new events to be consumed (period of inactivity) and no new punctuation happening.
3. Broker aborts the transaction after about 60000 ms  (default transaction timeout duration for the stream) , txnState=Ongoing -> txnState=PrepareAbort  ->  txnState=CompleteAbort
4. A new event comes in and hits producer fenced exception
 
Attached are the logs from the broker where a successful processing, along with transaction timeout and producer fencing is shown.
 
 

Issue Links

• is duplicated by: KAFKA-6906 Kafka Streams does not commit transactions if data is produced via wall-clock punctuation