[FLINK-26772] Application and Job Mode does not wait for job cleanup during shutdown

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.15.0
• Component/s: Runtime / Coordination
• Labels: auto-deprioritized-critical auto-deprioritized-major pull-request-available
• Assignee: Unassigned
• Reporter: Mika Naylor
• Created: 21/Mar/22 09:49
• Updated: 22/Aug/23 10:35

Description

We discovered that in Application Mode, when the application has completed, the cluster is shutdown even if there are ongoing resource cleanup events happening in the background. For example, if ha cleanup fails, further retries are not attempted as the cluster is shut down before this can happen.

 

We should also add a flag for the shutdown that will prevent further jobs from being submitted.

Issue Links

• is related to: FLINK-24491 ExecutionGraphInfo may not be archived when the dispatcher terminates
• is related to: FLINK-21008 Residual HA related Kubernetes ConfigMaps and ZooKeeper nodes when cluster entrypoint received SIGTERM in shutdown
• links to: GitHub Pull Request #19317

Sub-Tasks

• Make Job mode wait with cluster shutdown until the cleanup is done: Resolved
• Make Application mode wait with cluster shutdown until cleanup is done: Open