[KAFKA-14910] Consider cancelling ongoing alter / reset connector offsets requests when the connector is resumed

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: connect
• Assignee: Unassigned
• Reporter: Yash Mayya
• Created: 15/Apr/23 05:42
• Updated: 15/Apr/23 05:42

Description

See discussion here for more details - https://github.com/apache/kafka/pull/13465#discussion_r1164465874

The implementation for the PATCH /connectors/{connector}/offsets and DELETE /connectors/{connector}/offsets APIs is completely asynchronous and the check for whether the connector is stopped will only be made at the beginning of the request. 

If the connector is resumed while the alter / reset offsets request is being processed, this can lead to certain issues (especially with non-EoS source connectors). For sink connectors, admin client requests to alter / reset offsets for a consumer group will be rejected if the consumer group is active (i.e. when the connector tasks come up). For source connectors when exactly once support is enabled on the worker, we do a round of zombie fencing before the tasks are brought up and this will basically disable the transactional producer used to alter offsets (the transactional producer uses the transactional ID for task 0 of the connector). However, for source connectors when exactly once support is not enabled on the worker (this is the default), there are no such safeguards. We could potentially add some interruption logic that cancels ongoing alter / reset offset requests when a connector is resumed.