[KAFKA-6131] Transaction markers are sometimes discarded if txns complete concurrently

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.11.0.1, 1.0.0
• Fix Version/s: 0.11.0.2, 1.0.0, 1.1.0
• Component/s: core
• Assignee: Rajini Sivaram
• Reporter: Rajini Sivaram
• Created: 26/Oct/17 18:12
• Updated: 27/Oct/17 02:14
• Resolved: 27/Oct/17 02:14

Description

Concurrent tests being added under KAFKA-6096 for transaction coordinator fail to complete some transactions when multiple transactions are completed concurrently.

The problem is with the following code snippet - there are two very similar uses of concurrent map in TransactionMarkerChannelManager and the test fails because some transaction markers are discarded. getOrElseUpdate in scala maps are not atomic. The test passes consistently with one thread.

val markersQueuePerBroker: concurrent.Map[Int, TxnMarkerQueue] = new ConcurrentHashMap[Int, TxnMarkerQueue]().asScala
val brokerRequestQueue = markersQueuePerBroker.getOrElseUpdate(brokerId, new TxnMarkerQueue(broker))

Issue Links

• links to: GitHub Pull Request #4140