[FLINK-29379] Back (most of the) ExecutionConfig and CheckpointConfig by Configuration

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.17.0
• Component/s: API / DataStream
• Labels: pull-request-available
• Assignee: Piotr Nowojski
• Reporter: Timo Walther
• Created: 21/Sep/22 09:10
• Updated: 08/Nov/22 13:16
• Resolved: 08/Nov/22 13:15

Description

Not sure if this is a duplicate, but as this issue pops up over and over again, it might be time to discuss it here and fix it.

Currently, configuration is spread across instances of Configuration and POJOs (e.g. ExecutionConfig or CheckpointConfig). This makes it very tricky to handle configuration throughout the stack. The practice has shown that configuration might be passed, layered, merged, restricted, copied, filtered, etc. This is easy with the config option stack but very tricky with the existing POJOs. Esp. it is difficult to keep the two in sync or compare them.

Many locations reveal the current shortcoming. For example, org.apache.flink.table.planner.delegation.DefaultExecutor has a isCheckpointingEnabled() method simply because we cannot trust the Configuration object that is passed around. Same for checking if object reuse is enabled.

A solution is still up for discussion. Ideally, we deprecate ExecutionConfig and CheckpointConfig and advocate a pure config option based approach. Alternatively, we could do a hybrid approach similar to `TableConfig` (that is backed by config options but has setters for convenience). The latter approach would cause less deprecations in the API.

Issue Links

• is blocked by: FLINK-29807 Drop TypeSerializerConfigSnapshot and savepoint support from Flink versions < 1.8.0
• is related to: FLINK-3642 Disentangle ExecutionConfig
• is related to: FLINK-29309 Relax allow-client-job-configurations for Table API and parameters
• relates to: FLINK-13876 Remove ExecutionConfig field from PojoSerializer
• links to: GitHub Pull Request #21056
• links to: GitHub Pull Request #21245