Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-4387

KafkaConsumer will enter an infinite loop if the polling thread is interrupted, and either commitSync or committed is called

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.1
    • Fix Version/s: 0.10.2.0
    • Component/s: clients
    • Labels:
      None

      Description

      When the KafkaConsumer.commitSync method is called, the ConsumerNetworkClient.poll(RequestFuture<?> future) method will be called with a future that only finishes when the commit request completes, or the request times out.

      When the calling thread is interrupted, every call to the Selector underlying the ConsumerNetworkClient will return immediately, while thread interrupt state is not reset. The call to poll ends up looping until the request timeout, at which point it drops back out to ConsumerCoordinator.commitOffsetsSync which retries the request because TimeoutException is retriable. This repeats indefinitely.

      For the same reason as in https://issues.apache.org/jira/browse/KAFKA-4375, it is good if the KafkaConsumer can handle interrupts in a reasonable way, rather than having wakeup() be the only way to properly stop a consumer thread.

      I think making ConsumerNetworkClient.maybeTriggerWakeup() throw a WakeupException if the calling thread is interrupted makes sense, since an interrupted thread won't be making progress in polling due to the way Selector works, and KafkaConsumer users then don't have to handle wakeups and interrupts separately.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                srdo Stig Rohde Døssing
                Reporter:
                srdo Stig Rohde Døssing
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: