[KAFKA-14880] TransactionMetadata with producer epoch -1 should be expirable

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.0.0, 3.2.0, 3.3.0, 3.4.0
• Fix Version/s: 3.2.4, 3.1.3, 3.0.3, 3.4.1, 3.3.3
• Assignee: David Jacot
• Reporter: David Jacot
• Created: 05/Apr/23 08:11
• Updated: 06/Apr/23 07:18
• Resolved: 06/Apr/23 07:18

Description

We have seen the following error in logs:

"Mar 22, 2019 @ 21:57:56.655",Error,"kafka-0-0","transaction-log-manager-0","Uncaught exception in scheduled task 'transactionalId-expiration'","java.lang.IllegalArgumentException: Illegal new producer epoch -1

Investigations showed that it is actually possible for a transaction metadata object to still have -1 as producer epoch when it transitions to Dead.

When a transaction metadata is created for the first time (in handleInitProducerId), it has -1 as its producer epoch. Then a producer epoch is attributed and the transaction coordinator tries to persist the change. If the write fail for instance because there is an under min isr, the transaction metadata remains with its epoch as -1 forever or until the init producer id is retried.

This means that it is possible for transaction metadata to remain with -1 as producer epoch until it gets expired. At the moment, this is not allowed because we enforce a producer epoch greater or equals to 0 in prepareTransitionTo.

Issue Links

• links to: GitHub Pull Request #13499