[KAFKA-9749] TransactionMarkerRequestCompletionHandler should treat storage exceptions as retriable

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.5.0
• Fix Version/s: 2.5.0, 2.3.2, 2.4.2
• Component/s: core
• Assignee: Bob Barrett
• Reporter: Bob Barrett
• Created: 23/Mar/20 21:48
• Updated: 26/Jan/21 20:27
• Resolved: 24/Mar/20 21:56

Description

If `TransactionMarkerRequestCompletionHandler` handles a `KafkaStorageException`, it throws an IllegalStateException rather than retrying. This leaves the corresponding transactional ID in state PendingAbort, where it gets stuck, because any further EndTxn or InitProducerId call will fail with a CONCURRENT_TRANSACTIONS error. We should retry these errors when writing transaction markers.

Issue Links

• causes: KAFKA-8803 Stream will not start due to TimeoutException: Timeout expired after 60000milliseconds while awaiting InitProducerId
• links to: GitHub Pull Request #8336