Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3850

WorkerSinkTask should retry commits if woken up during rebalance or shutdown

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.10.0.0
    • 0.10.0.1, 0.10.1.0
    • connect
    • None

    Description

      We use consumer.wakeup() to interrupt long polls when we need to pause/resume partitions and when we shutdown sink tasks. The resulting WakeupException could be raised from the synchronous commit which we use in between rebalances and on shutdown. Since we don't currently catch this exception, we can fail to commit offsets, which typically results in duplicates. To fix this problem, we should catch the exception, retry the commit, and then rethrow it.

      Attachments

        Issue Links

          Activity

            People

              hachikuji Jason Gustafson
              hachikuji Jason Gustafson
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: