[KAFKA-5364] Producer attempts to send transactional messages before adding partitions to transaction

• Type: Sub-task
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.11.0.0
• Fix Version/s: 0.11.0.0
• Component/s: clients, core, producer
• Labels: exactly-once
• Assignee: Apurva Mehta
• Reporter: Apurva Mehta
• Created: 01/Jun/17 22:26
• Updated: 06/Jun/17 20:50
• Resolved: 06/Jun/17 20:50

Description

Due to a race condition between the sender thread and the producer.send(), the following is possible:

1. In KakfaProducer.doSend(), we add partitions to the transaction and then do accumulator.append.
2. In Sender.run(), we check whether there are transactional request. If there are, we send them and wait for the response.
3. If there aren't we drain the accumulator queue and send the produce requests.
4. The problem is that the sequence step 2, 1, 3 is entire possible. This means that we won't send the 'AddPartitions' request but yet try to send the produce data. Which results in a fatal error and requires the producer to close.

The solution is that in the accumulator.drain, we should check again if there are pending add partitions requests, and if so, don't drain anything.

Issue Links

• links to: GitHub Pull Request #3202
• links to: GitHub Pull Request #3231