[FLINK-10912] Configurable RocksDBStateBackend options factory

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Implemented
• Affects Version/s: 1.7.0
• Fix Version/s: 1.8.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available
• Assignee: Yun Tang
• Reporter: Thomas Weise
• Created: 16/Nov/18 18:51
• Updated: 15/Mar/19 14:22
• Resolved: 27/Feb/19 10:25

Description

RocksDB has a ton of configuration options and users should be able to reach those, when necessary. Currently there are a few predefined configurations that can be activated programmatically [1] [2]. This requires change to the pipeline construction code that's hard or impossible when using higher level API or frameworks such as Apache Beam. It would be good to support an options factory that can be specified as part of the state backend configuration in flink-conf.yaml.

[1] https://github.com/apache/flink/blob/release-1.5/flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/PredefinedOptions.java#L44

[2] https://stackoverflow.com/questions/44819521/how-to-set-rocksdbstatebackend-parameter-in-flink

Issue Links

• links to: GitHub Pull Request #7586