Kafka
  1. Kafka
  2. KAFKA-234

Make backoff time during consumer rebalance configurable

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.7.1
    • Fix Version/s: 0.7.1
    • Component/s: core
    • Labels:
      None

      Description

      We need to make backoff time during consumer rebalance directly configurable, instead of relying on zkSyncTime.

        Activity

        Hide
        Jun Rao added a comment -

        Patch submitted.

        Show
        Jun Rao added a comment - Patch submitted.
        Hide
        Neha Narkhede added a comment -

        +1

        Show
        Neha Narkhede added a comment - +1
        Hide
        Jun Rao added a comment -

        Thanks for the review. Committed.

        Show
        Jun Rao added a comment - Thanks for the review. Committed.
        Hide
        Guozhang Wang added a comment -

        Hi Jun,

        Just wondering why we want to make zkSyncTime as the default value of fetcherBackoffMs at the first place? These two configs do not seem related to me.

        Show
        Guozhang Wang added a comment - Hi Jun, Just wondering why we want to make zkSyncTime as the default value of fetcherBackoffMs at the first place? These two configs do not seem related to me.
        Hide
        Jun Rao added a comment -

        The reasoning is the following. A ZK follower can be up to zkSyncTime behind the ZK leader. Since a ZK client can do reads from any ZK server, it's possible for a rebalance to fail because one of the consumers reads stale data. Waiting for at least zkSyncTime allows the ZK follower to have enough time to sync up with its leader.

        Show
        Jun Rao added a comment - The reasoning is the following. A ZK follower can be up to zkSyncTime behind the ZK leader. Since a ZK client can do reads from any ZK server, it's possible for a rebalance to fail because one of the consumers reads stale data. Waiting for at least zkSyncTime allows the ZK follower to have enough time to sync up with its leader.

          People

          • Assignee:
            Jun Rao
            Reporter:
            Jun Rao
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development