[KAFKA-13782] Producer may fail to add the correct partition to transaction

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 3.2.0, 3.1.1
• Assignee: Jason Gustafson
• Reporter: Jason Gustafson
• Created: 30/Mar/22 01:39
• Updated: 06/Apr/22 05:20
• Resolved: 05/Apr/22 17:11

Description

In KAFKA-13412, we changed the logic to add partitions to transactions in the producer. The intention was to ensure that the partition is added in `TransactionManager` before the record is appended to the `RecordAccumulator`. However, this does not take into account the possibility that the originally selected partition may be changed if `abortForNewBatch` is set in `RecordAppendResult` in the call to `RecordAccumulator.append`. When this happens, the partitioner can choose a different partition, which means that the `TransactionManager` would be tracking the wrong partition.

I think the consequence of this is that the batches sent to this partition would get stuck in the `RecordAccumulator` until they timed out because we validate before sending that the partition has been added correctly to the transaction.

Note that KAFKA-13412 has not been included in any release, so there are no affected versions.

Thanks to alivshits for identifying the bug.

Issue Links

• links to: GitHub Pull Request #11995