[FLINK-31838] Move thread handling from DefaultMultipleComponentLeaderElectionService into DefaultLeaderElectionService

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.18.0
• Labels: pull-request-available
• Assignee: Matthias Pohl
• Reporter: Matthias Pohl
• Created: 18/Apr/23 11:14
• Updated: 14/Jun/23 11:23
• Resolved: 05/May/23 10:18

Description

While working on FLINK-31773 we noticed in the PR (see comment #1 and comment #2) that there's a different notion of responsibility of the thread handling between the FLINK-24038 and the pre-FLINK-24038 implementation of the leader election. The latter one (i.e. 
DefaultLeaderElectionService) doesn't need to deal with where onGrantLeadership and onRevokeLeadership are called because those are called in a separate thread in the driver. The new implementation (i.e. 
DefaultMultipleComponentLeaderElectionService) has to call the onGrantLeadership and onRevokeLeadership methods itself when registering new contenders. This would happen in the main thread of the LeaderContender which is not suitable. DefaultMultipleComponentLeaderElectionService solves this by introducing its own threadpool.

The separation of driver and contender in FLINK-31773 requires such a executionService in DefaultLeaderElectionService as well before going ahead with FLINK-31773.

Issue Links

• blocks: FLINK-31773 Separate DefaultLeaderElectionService.start(LeaderContender) into two separate methods for starting the driver and registering a contender
• relates to: FLINK-31878 Fix the wrong name of PauseOrResumeSplitsTask#toString in connector fetcher
• Testing discovered: FLINK-31995 DirectExecutorService doesn't follow the ExecutorService contract throwing a RejectedExecutionException in case it's already shut down
• links to: GitHub Pull Request #22422