[FLINK-25715] Application Mode: Add option to submit a failed job on application error

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.15.0
• Component/s: Client / Job Submission, Runtime / Coordination
• Labels: pull-request-available
• Assignee: David Morávek
• Reporter: David Morávek
• Created: 20/Jan/22 08:37
• Updated: 21/Mar/22 08:07
• Resolved: 24/Jan/22 15:24

Description

Currently in application mode, any exception happens in the application driver (before submitting an actual job) leads to a fail-over. These errors are usually not retryable and we don't have a good way of reporting them to the user.

We'll introduce a new config option `execution.submit-failed-job-on-application-error` that submits a failed job with the `$internal.pipeline.job-id` instead.

This is intended to be used in combination with `execution.shutdown-on-application-finish = false` to allow user to retrieve the information about the failed submission.

Issue Links

• is required by: FLINK-25206 Add config option to disable configurations in the user program
• relates to: FLINK-24113 Introduce option in Application Mode to disable shutdown
• links to: GitHub Pull Request #18416