[FLINK-15692] Enable limiting RocksDB memory consumption by default

• Type: Improvement
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.0
• Component/s: Runtime / State Backends
• Labels: pull-request-available
• Assignee: Yun Tang
• Reporter: Till Rohrmann
• Created: 20/Jan/20 15:47
• Updated: 27/Apr/21 18:06
• Resolved: 23/Jan/20 13:24

Description

Enable limiting RocksDB memory consumption by default. This means to change the default value of state.backend.rocksdb.memory.managed to true. This will change Flink's default behaviour for many users but I believe the benefits of a much more predictable memory behaviour of Flink will outweigh the downside of having fewer memory available.

Issue Links

• causes: FLINK-15749 Tune managed memory size for flink-benchmark jobs
• is related to: FLINK-15748 Set explict managed memory for RocksDBStateBenchmark
• relates to: FLINK-13980 FLIP-49 Unified Memory Configuration for TaskExecutors
• relates to: FLINK-7289 Memory allocation of RocksDB can be problematic in container environments
• links to: GitHub Pull Request #10920
• links to: GitHub Pull Request #10921