[KAFKA-3462] Allow SinkTasks to disable consumer offset commit

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.10.1.0
• Fix Version/s: 0.10.2.0
• Component/s: connect
• Assignee: Liquan Pei
• Reporter: Liquan Pei
• Created: 25/Mar/16 00:36
• Updated: 25/Feb/18 21:32
• Resolved: 15/Nov/16 00:24

Description

SinkTasks should be able to disable consumer offset commit if they manage offsets in the sink data store rather than using Kafka consumer offsets. For example, an HDFS connector might record offsets in HDFS to provide exactly once delivery. When the SinkTask is started or a rebalance occurs, the task would reload offsets from HDFS. In this case, disabling consumer offset commit will save some CPU cycles and network IOs.

Issue Links

• Is contained by: KAFKA-4161 Decouple flush and offset commits
• links to: GitHub Pull Request #1139