[FLINK-33297] FLIP-366: Support standard YAML for FLINK configuration

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.19.0
• Component/s: Runtime / Configuration
• Labels: 2.0-related pull-request-available

Release Note

Starting with Flink-1.19, Flink has officially introduced full support for the standard YAML 1.2 syntax (FLIP-366). The default configuration file has been changed to `config.yaml` and placed in the `conf/` directory. Users should directly modify this file to configure Flink.

If users want to use the legacy configuration file `flink-conf.yaml`, users just need to copy this file into the `conf/` directory. Once the legacy configuration file `flink-conf.yaml` is detected, Flink will prioritize using it as the configuration file.
And in the upcoming Flink 2.0, the `flink-conf.yaml` configuration file will no longer work.

More details could be found at Flink website (https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/config/#flink-configuration-file)

• Assignee: Junrui Li
• Reporter: Junrui Li
• Created: 18/Oct/23 03:14
• Updated: 05/Feb/24 02:31
• Resolved: 05/Feb/24 02:31

Description

Support standard YAML for FLINK configuration

Issue Links

• Blocked: FLINK-34114 Parse error while $internal.application.program-args contains '#' in Yarn/K8s Application Mode
• blocks: FLINK-33677 Remove flink-conf.yaml from flink dist
• causes: FLINK-34116 GlobalConfigurationTest.testInvalidStandardYamlFile fails
• relates to: FLINK-23620 Introduce proper YAML parsing to Flink's configuration

Sub-Tasks

• CLONE - Add a suffix to keys that violate YAML spec: Closed
• CLONE - Add checks to guarantee the non-deprecated options not conflicting with standard YAML: Closed
• Introduce standard YAML for flink configuration: Closed
• Introduce new Flink conf file "config.yaml" supporting standard YAML syntax: Closed
• Make "config.yaml" as the default Flink configuration file: Closed
• Extend BashJavaUtils to Support Reading and Writing Standard YAML Files: Closed
• Update flink-docker's Dockerfile and docker-entrypoint.sh to use BashJavaUtils for Flink configuration management: Closed
• Introduce a migration tool to transfer legacy config file to new config file: Closed
• Document FLIP-366: Support standard YAML for FLINK configuration: Closed