[KAFKA-16570] FenceProducers API returns "unexpected error" when successful

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.8.0, 3.7.1
• Assignee: Edoardo Comar
• Reporter: Justine Olshan
• Created: 16/Apr/24 21:49
• Updated: 14/Jun/24 15:12
• Resolved: 13/Jun/24 13:04

Description

When we want to fence a producer using the admin client, we send an InitProducerId request.

There is logic in that API to fence (and abort) any ongoing transactions and that is what the API relies on to fence the producer. However, this handling also returns CONCURRENT_TRANSACTIONS. In normal usage, this is good because we want to actually get a new producer ID and want to retry until the the ID is supplied or we time out.  https://github.com/apache/kafka/blob/5193eb93237ba9093ae444d73a1eaa2d6abcc9c1/core/src/main/scala/kafka/coordinator/transaction/TransactionCoordinator.scala#L170 
https://github.com/apache/kafka/blob/a3dcbd4e28a35f79f75ec1bf316ef0b39c0df164/clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionManager.java#L1322 

In the case of fence producer, we don't retry and instead we have no handling for concurrent transactions and log a message about an unexpected error.
https://github.com/apache/kafka/blob/a3dcbd4e28a35f79f75ec1bf316ef0b39c0df164/clients/src/main/java/org/apache/kafka/clients/admin/internals/FenceProducersHandler.java#L112 

This is not unexpected though and the operation was successful. We should just swallow this error and treat this as a successful run of the command. 

Issue Links

• is duplicated by: KAFKA-14657 Admin.fenceProducers fails when Producer has ongoing transaction - but Producer gets fenced
• links to: GitHub Pull Request #16229