[KAFKA-3850] WorkerSinkTask should retry commits if woken up during rebalance or shutdown

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.10.0.0
• Fix Version/s: 0.10.0.1, 0.10.1.0
• Component/s: KafkaConnect
• Assignee: Jason Gustafson
• Reporter: Jason Gustafson
• Created: 16/Jun/16 03:01
• Updated: 19/Jun/16 22:32
• Resolved: 19/Jun/16 22:32

Description

We use consumer.wakeup() to interrupt long polls when we need to pause/resume partitions and when we shutdown sink tasks. The resulting WakeupException could be raised from the synchronous commit which we use in between rebalances and on shutdown. Since we don't currently catch this exception, we can fail to commit offsets, which typically results in duplicates. To fix this problem, we should catch the exception, retry the commit, and then rethrow it.

Issue Links

• links to: GitHub Pull Request #1511