Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-3850

WorkerSinkTask should retry commits if woken up during rebalance or shutdown

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0.0
    • Fix Version/s: 0.10.0.1, 0.10.1.0
    • Component/s: KafkaConnect
    • Labels:
      None

      Description

      We use consumer.wakeup() to interrupt long polls when we need to pause/resume partitions and when we shutdown sink tasks. The resulting WakeupException could be raised from the synchronous commit which we use in between rebalances and on shutdown. Since we don't currently catch this exception, we can fail to commit offsets, which typically results in duplicates. To fix this problem, we should catch the exception, retry the commit, and then rethrow it.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                hachikuji Jason Gustafson
                Reporter:
                hachikuji Jason Gustafson
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: