[FLINK-11525] Add option to manually set job ID

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.7.0
• Component/s: Command Line Client, Runtime / REST
• Assignee: Unassigned
• Reporter: Ufuk Celebi
• Created: 05/Feb/19 10:54
• Updated: 09/Oct/20 16:10
• Resolved: 09/Oct/20 16:10

Description

When submitting Flink jobs programmatically it is desirable to have the option to manually set the job ID in order to have idempotent job submissions. This simplifies failure handling on the user side as duplicate submissions will be rejected by Flink. In general allowing to manually set the job ID can be beneficial for third party tooling.

The default behavior should not be altered. The following job submission entry points should be extended to allow to specify this option:
1. REST API
2. StandaloneJobClusterEntrypoint
3. CLI

Note that for 2. FLINK-10921 already suggested to allow to configure the job ID manually.

If there is agreement to have this feature, I'll go ahead and create sub tasks for the mentioned entry points (and provide the implementation for 1. and 2.).

Sub-Tasks

• Add option to manually set job ID for job submissions via REST API: Closed
• Add option to manually set job ID in StandaloneJobClusterEntryPoint: Closed
• Add option to manually set job ID in CLI: Closed