[KAFKA-14725] Improve cancellation semantics for connector tasks

• Type: Improvement
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Component/s: connect
• Assignee: Chris Egerton
• Reporter: Chris Egerton
• Created: 15/Feb/23 20:58
• Updated: 29/Jan/24 20:12

Description

This came about during discussion on https://github.com/apache/kafka/pull/13208/, which addressed KAFKA-5756.

 

Right now, we make some effort to disable and shut down tasks that have been scheduled for shutdown but taken longer than the graceful shutdown timeout period.

The logic for performing this disablement is contained in the cancel method for the WorkerTask and its subclasses (at the time of writing, that would be the AbstractWorkerSourceTask, WorkerSourceTask, ExactlyOnceWorkerSourceTask, and WorkerSinkTask classes). Right now we don't do much to interrupt in-progress operations, which may lead to zombie tasks lying around on a worker that have not yet relinquished resources like Kafka clients, file descriptors, or database connections despite being scheduled for shutdown.

We can and should make the cancellation logic for tasks more stringent, including but not limited to:

• Interrupting the work thread for the task
• Interrupting any in-progress offset commits
• Preemptively shutting down any Kafka clients created for use by the task

Issue Links

• is duplicated by: KAFKA-15252 Task is not stopped until the poll interval passes in case of task restarting.
• links to: GitHub Pull Request #14316