Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-10122

Consumer should allow heartbeat during rebalance as well

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Today we disable heartbeats if the state != MemberState.STABLE. And if a rebalance failed we set the state to UNJOINED. In the old API poll(long) it is okay since we always try to complete the rebalance successfully within the same call, so we would not be in UNJOINED or REBALANCING for a very long time.

      But with the new poll(Duration) we may actually return while we are still in UNJOINED or REBALANCING and it may take some time (smaller than max.poll.interval but larger than session.timeout) before the next poll call, and since heartbeat is disabled during this period of time we could be kicked by the coordinator.

      The proposal I have is

      1) allow heartbeat to be sent during REBALANCING as well.
      2) when join/sync response has retriable error, do not set the state to UNJOINED but stay with REBALANCING.

        Attachments

          Activity

            People

            • Assignee:
              guozhang Guozhang Wang
              Reporter:
              guozhang Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: