[FLINK-20695] Zookeeper node under leader and leaderlatch is not deleted after job finished

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.9.0, 1.11.3, 1.12.0, 1.13.0
• Fix Version/s: 1.14.0, 1.13.1, 1.12.5
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Release Note: The `HighAvailabilityServices` have received a new method `cleanupJobData` which can be implemented in order to clean up job related HA data after a given job has terminated.
• Assignee: Yi Tang
• Reporter: lidesheng
• Created: 21/Dec/20 08:34
• Updated: 11/Dec/23 09:09
• Resolved: 20/May/21 15:55

Description

I used flink 1.11 in standalone cluster mode for batch job. The enviornment was configured as zookeeper HA mode.

After job was commited, flink runtime created nodes under /flink/default/leader and /flink/default/leaderlatch with job id.  Though jobs were finished, these nodes  were remaining in zookeeper path forever. 

After a period of running, more and more jobs had been executed and there were a greate number of nodes under /flink/default/leader and slowed down the performance of zookeeper. Why not delete the nodes after job finished? Flink runtime could get job status by listeners and delete the leader nodes for job immidiately.

Issue Links

• causes: FLINK-22721 Breaking HighAvailabilityServices interface by adding new method
• is related to: FLINK-21942 KubernetesLeaderRetrievalDriver not closed after terminated which lead to connection leak
• relates to: FLINK-22704 ZooKeeperHaServicesTest.testCleanupJobData failed
• relates to: FLINK-22636 Group job specific ZooKeeper HA services under common jobs/<JobID> zNode
• links to: GitHub Pull Request #15561