[FLINK-18677] ZooKeeperLeaderRetrievalService does not invalidate leader in case of SUSPENDED connection

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.1, 1.11.1, 1.12.0
• Fix Version/s: 1.10.2, 1.11.2, 1.12.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Matthias Pohl
• Reporter: Till Rohrmann
• Created: 22/Jul/20 21:16
• Updated: 28/Aug/21 11:16
• Resolved: 05/Aug/20 09:38

Description

The ZooKeeperLeaderRetrievalService does not invalidate the leader if the ZooKeeper connection gets SUSPENDED. This means that a TaskManager won't cancel its running tasks even though it might miss a leader change. I think we should at least make it configurable whether in such a situation the leader listener should be informed about the lost leadership. Otherwise, we might run into the situation where an old and a newly recovered instance of a Task can run at the same time.

Issue Links

• relates to: FLINK-18451 Flink HA on yarn may appear TaskManager double running when HA is restored
• links to: GitHub Pull Request #13055