[FLINK-27060] [FLIP-256] Support Job Dynamic Parameter With Flink Rest Api - ASF JIRA

Agile Board

Rank to Top

Rank to Bottom

Attach files

Attach Screenshot

Bulk Copy Attachments

Bulk Move Attachments

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Labels

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: New Feature
Status: Closed
Priority: Major
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 1.17.0
Component/s: Runtime / REST
Labels:
None

Description

Background

Users want to submit job via Flink REST API instead of Flink CLI which is more heavy-weight in certain scenarios, for example, a lightweight data processing workflow system that has Flink related systems.

Currently, the /jars/:jarid/run API (https://nightlies.apache.org/flink/flink-docs-master/docs/ops/rest_api/#jars-jarid-run) only supports a few selected Flink config options listed in the doc (parallelism、savepoint path and allow non-restored state), which is insufficient for practical use.

Proposed Changes

Extending the /jars/:jarid/run API with an additional request body parameter "configs", which is a map of flink configuration option-value pairs set by users.

For backward compatibility, we can retain the existing body parameters like "allowNonRestoredState", and when there's conflicting configurations, let the values set explicitly with existing body parameters take higher precedence over the values set by configs.