Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2792

KafkaConsumer.close() can block unnecessarily due to leave group waiting for a reply

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • None
    • 0.9.0.0
    • consumer
    • None

    Description

      The current implementation of close() waits for a response to LeaveGroup. However, if we have an outstanding rebalance in the works, this can cause the close() operation to have to wait for the entire rebalance process to complete, which is annoying since the goal is to get rid of the consumer object anyway. This is at best surprising and at worst can cause unexpected bugs due to close() taking excessively long – this was found due to exceeding timeouts unexpectedly causing other operations in Kafka Connect to timeout.

      Waiting for a response isn't necessary since as soon as the data is in the TCP buffer, it'll be delivered to the broker. The client doesn't benefit at all from seeing the close group. So we can instead just always send the request

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            ewencp Ewen Cheslack-Postava
            ewencp Ewen Cheslack-Postava
            Guozhang Wang Guozhang Wang
            Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment