[FLINK-4764] Introduce config options

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.1.2
• Fix Version/s: 1.2.0
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 07/Oct/16 12:09
• Updated: 28/Feb/19 13:21
• Resolved: 10/Oct/16 13:48

Description

It is a bit unorthodox to start a discussion via a pull request, but this suggestion is best motivated via some code.

I suggest to move away from the current model with `ConfigConstants` and move to a model where an `Option` object describes a configuration option completely, with default value, fallback keys.

Advantages

• Much simpler / easier access to values that have deprecated keys
• Not possible to accidentally overlook deprecated keys
• Key and default values are grouped together in the definition
• Clearly states the expected type value for each config key (string, int, etc).
• We can improve this even further to include the description and auto-generate the config docs

Example

Simple option:

Option<String> TASK_MANAGER_TMP_DIRS = new Option<>(
                    "taskmanager.tmp.dirs",                 // config key
                    System.getProperty("java.io.tmpdir"));  // default value

Option with multiple deprecated keys:

Option<String> HA_CLUSTER_ID = new Option<>(
                    "high-availability.cluster-id",      // config key
                    null,                                // no default value
                    "high-availability.zookeeper.path.namespace",  // latest deprecated key
                    "recovery.zookeeper.path.namespace");         // even earlier deprecated key

Get a config value, this automatically checks deprecated keys and default values:

final String zkQuorum = configuration.getValue(ConfigOptions.HA_ZOOKEEPER_QUORUM);
final long connTimeout = configuration.getInteger(ConfigOptions.HA_ZOOKEEPER_CONN_TIMEOUT);

Multiple Options classes

To avoid having one huge class (like `ConfigConstants`), we can easily split this up into `TaskManagerOptions`, `JobManagerOptions`, `ZooKeeperOptions`, etc.

Issue Links

• supercedes: FLINK-1936 Normalize keys and default values in ConfigConstants.java
• links to: GitHub Pull Request #2605