Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-8420

Graceful handling when consumer switches from subscribe to manual assign

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: consumer
    • Labels:
      None

      Description

      Today if a consumer switches between subscribe (and hence relies on group rebalance to get assignment) and manual assign, it may cause unnecessary rebalances. For example:

      1. consumer.subscribe();
      2. consumer.poll(); // join-group request sent, returns empty because poll timeout
      3. consumer.unsubscribe();
      4. consumer.assign(..);
      5. consumer.poll(); // sync-group request received, and the assigned partitions does not match the current subscription-state. In this case it will tries to re-join which is not necessary.

      In the worst case (i.e. leader keep sending incompatible assignment), this would case the consumer to fall into endless re-joins.

      Although it is not a very common usage scenario, it still worth being better handled than the status-quo.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              guozhang Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: