Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-2792

KafkaConsumer.close() can block unnecessarily due to leave group waiting for a reply

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0.0
    • Component/s: consumer
    • Labels:
      None

      Description

      The current implementation of close() waits for a response to LeaveGroup. However, if we have an outstanding rebalance in the works, this can cause the close() operation to have to wait for the entire rebalance process to complete, which is annoying since the goal is to get rid of the consumer object anyway. This is at best surprising and at worst can cause unexpected bugs due to close() taking excessively long – this was found due to exceeding timeouts unexpectedly causing other operations in Kafka Connect to timeout.

      Waiting for a response isn't necessary since as soon as the data is in the TCP buffer, it'll be delivered to the broker. The client doesn't benefit at all from seeing the close group. So we can instead just always send the request

        Attachments

          Activity

            People

            • Assignee:
              ewencp Ewen Cheslack-Postava
              Reporter:
              ewencp Ewen Cheslack-Postava
              Reviewer:
              Guozhang Wang
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: