[FLINK-27060] [FLIP-256] Support Job Dynamic Parameter With Flink Rest Api

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Component/s: Runtime / REST
• Assignee: ConradJam
• Reporter: Zhanghao Chen
• Created: 05/Apr/22 13:38
• Updated: 24/Nov/22 09:00
• Resolved: 15/Nov/22 08:57

Description

Background

Users want to submit job via Flink REST API instead of Flink CLI which is more heavy-weight in certain scenarios, for example, a lightweight data processing workflow system that has Flink related systems.

Currently, the /jars/:jarid/run API (https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jars-jarid-run) only supports a few selected Flink config options listed in the doc (parallelism、savepoint path and allow non-restored state), which is insufficient for practical use.

Proposed Changes

Extending the /jars/:jarid/run API with an additional request body parameter "configs", which is a map of flink configuration option-value pairs set by users.

For backward compatibility, we can retain the existing body parameters like "allowNonRestoredState", and when there's conflicting configurations, let the values set explicitly with existing body parameters take higher precedence over the values set by configs.

Issue Links

• is duplicated by: FLINK-28973 Extending /jars/:jarid/plan API to support setting Flink configs

Sub-Tasks

• Jar Run Rest Handler Support Flink Configuration: Closed
• Update Flink doc: Closed