Major Description
When the final cleanup for a sink task is initiated, the framework first calls stop() on the task, and then closes the consumer for the task. Closing the consumer has the side effect of triggering the onPartitionsRevoked method of its ConsumerRebalanceListener, which in turn causes the framework to call WorkerSinkTask::closePartitions, which in turn calls WorkerSinkTask::commitOffsets, and finally, in turn calls SinkTask:preCommit.
Calling SinkTask:preCommit after SinkTask::stop is likely to cause errors with tasks as they should be performing resource cleanup during stop, and the current documentation on the SinkTask lifecycle makes no mention of anything happening after tasks are stopped.
The framework already ensures that offsets are committed for tasks after the last call to SinkTask:put has been made, so the offset commit after SinkTask::stop has already been invoked can and should be removed with no compromise of existing delivery guarantees provided by the framework.
Attachments
Issue Links
- links to