Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4303

KafkaConsumer blocks unnecessarily in commitSync()

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.10.1.0
    • Component/s: consumer
    • Labels:
      None

      Description

      This appears to be a regression caused by the KIP-62 patch. It is possible that we end up blocking unnecessarily in NetworkClient.poll() with no requests pending. The reason is that ConsumerNetworkClient.poll(RequestFuture) does not actually verify that the request has been sent prior to calling NetworkClient.poll(). This was not as much of a problem previously because the maximum timeout was always bounded by the heartbeat interval, which is typically small.

      This appears to be the cause of hanging builds that some people may have experienced. In particular, I have seen ConsumerBounceTest hang because of this problem.

      Note that another reason we can block indefinitely in commitSync() is coordinator discovery. We do not attempt to fix this here.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                hachikuji Jason Gustafson
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: