[KAFKA-14972] Make KafkaConsumer usable in async runtimes

• Type: Wish
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Component/s: consumer
• Labels: needs-kip
• Assignee: Unassigned
• Reporter: Erik van Oosten
• Created: 07/May/23 18:37
• Updated: 05/Aug/23 18:27
• Resolved: 05/Aug/23 18:07

Description

KafkaConsumer contains a check that rejects nested invocations from different threads (method acquire). For users that use an async runtime, this is an almost impossible requirement. Examples of async runtimes that are affected are Kotlin co-routines (see KAFKA-7143) and Zio.

It should be possible for a thread to pass on its capability to access the consumer to another thread. See KIP-944 for a proposal and https://github.com/apache/kafka/pull/13914 for an implementation.

Issue Links

• links to: GitHub Pull Request #13914