[KAFKA-8304] Connect susceptible to deadlock while registering REST extensions

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 2.2.0, 2.1.1, 2.0.2, 2.3.0, 2.1.2, 2.2.1
• Fix Version/s: 2.0.2, 2.3.0, 2.1.2, 2.2.1
• Component/s: connect
• Assignee: Chris Egerton
• Reporter: Chris Egerton
• Created: 29/Apr/19 17:18
• Updated: 07/May/19 22:53
• Resolved: 07/May/19 22:53

Description

As part of KAFKA-7503, the ConnectClusterStateImpl class was altered to use a HerderProvider instance instead of a Herder. However, the Connect RestServer registers REST extensions before a herder is given to that HerderProvider, so any extensions that invoke, e.g., ConnectClusterState.connector() in their register(...) method end up in a deadlock that eventually causes Connect startup to fail with the error message "Timed out waiting for herder to be initialized."

If possible, the HerderProvider used for ConnectClusterStateImpl instances given to REST extensions should be supplied with a Herder before those extensions are registered. If that isn't feasible, another option could be to install Connect REST extensions on a separate thread so that they don't block the Connect startup process and eventual call of HerderProvider.setHerder(...).

Issue Links

• links to: GitHub Pull Request #6651