Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16599

LegacyConsumer: Always await async commit callbacks in commitSync and close

    XMLWordPrintableJSON

Details

    • Task
    • Status: Patch Available
    • Major
    • Resolution: Unresolved
    • None
    • 3.8.0
    • clients, consumer
    • None

    Description

      The javadoc for KafkaConsumer.commitSync says:

      Note that asynchronous offset commits sent previously with the {@link #commitAsync(OffsetCommitCallback)}
      (or similar) are guaranteed to have their callbacks invoked prior to completion of this method.
      

      This is not always true in the legacy consumer, when the set of offsets is empty, the execution of the commit callback is not always awaited. There are also various races possible that can avoid callback handler execution.

      Similarly, there is code in the legacy consumer to await the completion of the commit callback before closing, however, the code doesn't cover all cases and the behavior is therefore inconsistent. While the Javadoc doesn't explicitly promise callback execution, it promises "completing commits", which one would reasonably expect to include callback execution. Either way, the current behavior of the legacy consumer is inconsistent.

      Attachments

        Activity

          People

            lucasbru Lucas Brutschy
            lucasbru Lucas Brutschy
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated: