[KAFKA-15657] Unexpected errors when producing transactionally in 3.6

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.6.0
• Component/s: producer
• Assignee: Unassigned
• Reporter: Travis Bischel
• Created: 20/Oct/23 00:46
• Updated: 23/Oct/23 20:58
• Resolved: 23/Oct/23 20:58

Description

In loop-testing the franz-go client, I am frequently receiving INVALID_RECORD (which I created a separate issue for), and INVALID_TXN_STATE and UNKNOWN_SERVER_ERROR.

INVALID_TXN_STATE is being returned even though the partitions have been added to the transaction (AddPartitionsToTxn). Nothing about the code has changed between 3.5 and 3.6, and I have loop-integration-tested this code against 3.5 thousands of times. 3.6 is newly - and always - returning INVALID_TXN_STATE. If I change the code to retry on INVALID_TXN_STATE, I eventually quickly (always) receive UNKNOWN_SERVER_ERROR. In looking at the broker logs, the broker indicates that sequence numbers are out of order - but (a) I am repeating requests that were in order (so something on the broker got a little haywire maybe? or maybe this is due to me ignoring invalid_txn_state?), and I am not receiving OUT_OF_ORDER_SEQUENCE_NUMBER, I am receiving UNKNOWN_SERVER_ERROR.

I think the main problem is the client unexpectedly receiving INVALID_TXN_STATE, but a second problem here is that OOOSN is being mapped to USE on return for some reason.