[KAFKA-17105] Unnecessary connector restarts after being newly created

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.0.0, 3.0.1, 3.2.0, 3.1.1, 3.3.0, 3.0.2, 3.1.2, 3.2.1, 3.4.0, 3.2.2, 3.2.3, 3.3.1, 3.3.2, 3.5.0, 3.4.1, 3.6.0, 3.5.1, 3.5.2, 3.7.0, 3.6.1, 3.6.2, 3.8.0, 3.7.1, 3.9.0
• Fix Version/s: 3.9.0
• Component/s: connect
• Assignee: Chris Egerton
• Reporter: Chris Egerton
• Created: 09/Jul/24 22:37
• Updated: 17/Jul/24 18:33
• Resolved: 17/Jul/24 18:33

Description

When a connector is created, it may be restarted unnecessarily immediately after it is first started by the worker to which it has been assigned:

1. Connector config is written to the config topic
2. A worker reads the new record from the config topic, and adds the connector to its connectorConfigUpdates field (see here)
3. Another worker has already seen the new connector config and triggered a rebalance; this worker participates in the ensuing rebalance (see here) and is assigned the connector
4. After the rebalance is over (see here), the worker starts all of the connectors it has been newly-assigned (see here and here)
5. Once finished with that, the worker checks for new connector configs (see here) and restarts all connectors in the connectorConfigUpdates field (see here).

Issue Links

• links to: GitHub Pull Request #16581