[FLINK-23946] Application mode fails fatally when being shut down

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.12.5, 1.13.2
• Fix Version/s: 1.14.4, 1.15.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available stale-assigned
• Assignee: David Morávek
• Reporter: Till Rohrmann
• Created: 24/Aug/21 12:48
• Updated: 01/Feb/22 14:31
• Resolved: 01/Feb/22 14:31

Description

The application mode fails fatally when being shut down (e.g. if the Dispatcher loses its leadership). The problem seems to be that the ApplicationDispatcherBootstrap cancels the applicationExecutionTask and applicationCompletionFuture that can trigger the execution of the fatal exception handler in the handler of the applicationCompletionFuture.

I suggest to only call the fatal exception handler if an unexpected exception occurs in the applicationCompletionFuture callback.

Issue Links

• causes: FLINK-25271 ApplicationDispatcherBootstrapITCase. testDispatcherRecoversAfterLosingAndRegainingLeadership failed on azure
• is blocked by: FLINK-24038 DispatcherResourceManagerComponent fails to deregister application if no leading ResourceManager
• is related to: FLINK-25235 Re-enable ZooKeeperLeaderElectionITCase#testJobExecutionOnClusterWithLeaderChange
• links to: GitHub Pull Request #17000
• links to: GitHub Pull Request #18066