[KAFKA-16227] Console consumer fails with `IllegalStateException`

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.7.0
• Fix Version/s: 3.8.0
• Component/s: clients, consumer
• Labels: kip-848-client-support
• Assignee: Bruno Cadonna
• Reporter: David Jacot
• Created: 06/Feb/24 15:40
• Updated: 14/Mar/24 14:15
• Resolved: 12/Mar/24 10:04

Description

I have seen a few occurrences like the following one. There is a race between the background thread and the foreground thread. I imagine the following steps:

• quickstart-events-2 is assigned by the background thread;
• the foreground thread starts the initialization of the partition (e.g. reset offset);
• quickstart-events-2 is removed by the background thread;
• the initialization completes and quickstart-events-2 does not exist anymore.

 

[2024-02-06 16:21:57,375] ERROR Error processing message, terminating consumer process:  (kafka.tools.ConsoleConsumer$)
java.lang.IllegalStateException: No current assignment for partition quickstart-events-2
	at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignedState(SubscriptionState.java:367)
	at org.apache.kafka.clients.consumer.internals.SubscriptionState.updateHighWatermark(SubscriptionState.java:579)
	at org.apache.kafka.clients.consumer.internals.FetchCollector.handleInitializeSuccess(FetchCollector.java:283)
	at org.apache.kafka.clients.consumer.internals.FetchCollector.initialize(FetchCollector.java:226)
	at org.apache.kafka.clients.consumer.internals.FetchCollector.collectFetch(FetchCollector.java:110)
	at org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.collectFetch(AsyncKafkaConsumer.java:1540)
	at org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.pollForFetches(AsyncKafkaConsumer.java:1525)
	at org.apache.kafka.clients.consumer.internals.AsyncKafkaConsumer.poll(AsyncKafkaConsumer.java:711)
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:874)
	at kafka.tools.ConsoleConsumer$ConsumerWrapper.receive(ConsoleConsumer.scala:473)
	at kafka.tools.ConsoleConsumer$.process(ConsoleConsumer.scala:103)
	at kafka.tools.ConsoleConsumer$.run(ConsoleConsumer.scala:77)
	at kafka.tools.ConsoleConsumer$.main(ConsoleConsumer.scala:54)
	at kafka.tools.ConsoleConsumer.main(ConsoleConsumer.scala) 

 

 

Issue Links

• is related to: KAFKA-16290 Investigate propagating subscription state updates via queues
• links to: GitHub Pull Request #15491