[KAFKA-7961] Handle subscription changes with a rebalance in progress

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.3.0
• Component/s: consumer
• Assignee: Jose Armando Garcia Sancio
• Reporter: Jason Gustafson
• Created: 20/Feb/19 17:51
• Updated: 23/Feb/19 04:51
• Resolved: 23/Feb/19 04:51

Description

Due to wakeups or poll timeouts, it is possible to have a subscription changed while a rebalance is in progress. This can lead to an illegal state error such as the following if some of the assigned partitions no longer match the subscription:

java.lang.IllegalArgumentException: Assigned partition foo-0 for non-subscribed topic; subscription is [bar]
	at org.apache.kafka.clients.consumer.internals.SubscriptionState.assignFromSubscribed(SubscriptionState.java:192)
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:249)
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:410)
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:344)
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:344)
	at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1226)
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1191)
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1176)

Rather than requiring the assignment received from a rebalance to match the subscription, we should just request a rebalance.

Issue Links

• links to: GitHub Pull Request #6304