[FLINK-16835] Replace TableConfig with Configuration

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Done
• Fix Version/s: 1.16.0
• Component/s: Table SQL / API
• Labels: auto-deprioritized-major
• Assignee: Marios Trivyzas
• Reporter: Timo Walther
• Created: 27/Mar/20 16:33
• Updated: 11/Apr/22 08:39
• Resolved: 11/Apr/22 07:56

Description

In order to allow reading and writing of configuration from a file or string-based properties. We should consider removing TableConfig and fully rely on a Configuration-based object with ConfigOptions.

This effort was partially already started which is why TableConfig.getConfiguration exists.

However, we should clarify if we would like to have control and traceability over layered configurations such as flink-conf,yaml < StreamExecutionEnvironment < TableEnvironment < Query. Maybe the Configuration class is not the right abstraction for this.

jark, twalthr, and fhueske discussed the configuration options (see comments below) and concluded with the following design:

  public static final ConfigOption<Duration> IDLE_STATE_RETENTION =
      key("table.exec.state.ttl")
          .durationType()
          .defaultValue(Duration.ofMillis(0));

  public static final ConfigOption<Integer> MAX_LENGTH_GENERATED_CODE =
      key("table.generated-code.max-length")
          .intType()
          .defaultValue(64000);

  public static final ConfigOption<String> LOCAL_TIME_ZONE =
      key("table.local-time-zone")
          .stringType()
          .defaultValue(ZoneId.systemDefault().toString());

Note: The following TableConfig options are not preserved:

• nullCheck: Flink will automatically enable null checks based on the table schema (NOT NULL property)
• decimalContext: this configuration is only used by the legacy planner which will be removed in one of the next releases
• maxIdleStateRetention: is automatically derived as 1.5* idleStateRetention until StateTtlConfig is fully supported (at which point only a single parameter is required).

Issue Links

• Blocked: FLINK-16875 Tracking replace QueryConfig and TableConfig implementation with ConfigOption

Sub-Tasks

• Make TableConfig options can be configured by string-based Configuration: Closed
• Deprecate unused options in TableConfig: Closed
• Remove getTableConfig from ExecNodeConfiguration: Closed
• Drop the unused options in TableConfig: Closed
• Blink planner should only use ReadableConfig instead of TableConfig: Closed
• Add convenient TableConfig.set: Closed
• Propagate executor config to TableConfig: Closed
• Remove 'table.planner' table config option: Closed
• Remove usages of deprecated TableConfig options: Closed
• Replace TableConfig with ReadableConfig where possible: Closed
• Replace TableConfig.getConfiguration().get(Option) with TableConfig.get(Option): Closed
• Deprecate TableConfig ctor and use getDefault() instead: Closed
• Replace TableConfig.getConfiguration().get(Option) with TableConfig.get(Option) (python): Closed
• Remove PlannerConfig: Closed
• Cleanup code around TableConfig/ReadableConfig: Closed
• Update docs regarding EnvironmentSettings / TableConfig: Closed