Details
-
Sub-task
-
Status: Resolved
-
Blocker
-
Resolution: Fixed
-
4.0.0
-
None
-
None
-
None
Description
With https://issues.apache.org/jira/browse/KAFKA-14563 there are two cases: either a transaction is already added to the transaction and we continue producing/committing offsets, or if it is not we add it with an interbroker call. With TV2, we can tell if a transaction was added in this or a previous transaction by checking the epoch. However, with the upgrade path, the epoch check doesn't work because the first TV2 transaction will have the same epoch as the last TV0 transaction.
If a transaction is in PrepareAbort/PrepareCommit, the producer can return a EndTxn response and is free to start the next transaction. It is possible that this new transaction will produce and commit offsets without informing the coordinator if the partitions are the same as the last transaction. Then on EndTxn, we can get fenced since the epoch is the same as the previous epoch (ie not a retry).
In other scenarios (like there are new partitions)we may also see InvalidStateTransitions.
Upgrades should not require producer restarts or lose guarantees so we should fix this.
Attachments
Issue Links
- links to