Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-14048 The Next Generation of the Consumer Rebalance Protocol
  3. KAFKA-16224

Fix handling of deleted topic when auto-committing before revocation

    XMLWordPrintableJSON

Details

    Description

      Current logic for auto-committing offsets when partitions are revoked will retry continuously when getting UNKNOWN_TOPIC_OR_PARTITION, leading to the member not completing the revocation in time. We should consider this as an indication of the topic being deleted, and in the context of committing offsets to revoke partitions, we should abort the commit attempt and move on to complete and ack the revocation (meaning that we would effectively consider UnknownTopicOrPartitionException as non-retriable, even though it extends RetriableException, only when committing offsets before revocation as part of this task)
      Note that legacy coordinator behaviour around this seems to be the same as the new consumer currently has, tracked with a related issue given that it would require a separate fix for the legacy consumer.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. KAFKA-16235 auto commit still causes delays due to retriable UNKNOWN_TOPIC_OR_PARTITION

          • Major - Major loss of function.
          • Open
          links to

          Web Link GitHub Pull Request #15581

          Activity

            People

              cadonna Bruno Cadonna
              lianetm Lianet Magrans
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: