[KAFKA-8420] Graceful handling when consumer switches from subscribe to manual assign

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: consumer
• Assignee: Matthew de Detrich
• Reporter: Guozhang Wang
• Created: 23/May/19 20:17
• Updated: 05/Feb/23 02:46

Description

Today if a consumer switches between subscribe (and hence relies on group rebalance to get assignment) and manual assign, it may cause unnecessary rebalances. For example:

1. consumer.subscribe();
2. consumer.poll(); // join-group request sent, returns empty because poll timeout
3. consumer.unsubscribe();
4. consumer.assign(..);
5. consumer.poll(); // sync-group request received, and the assigned partitions does not match the current subscription-state. In this case it will tries to re-join which is not necessary.

In the worst case (i.e. leader keep sending incompatible assignment), this would case the consumer to fall into endless re-joins.

Although it is not a very common usage scenario, it still worth being better handled than the status-quo.