[FLINK-11313] [checkpoint] Introduce LZ4 compression for keyed state in full checkpoints and savepoints

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Runtime / Checkpointing
• Labels: auto-deprioritized-major auto-deprioritized-minor auto-unassigned pull-request-available
• Assignee: Unassigned
• Reporter: Yun Tang
• Created: 14/Jan/19 08:35
• Updated: 15/Dec/23 04:00

Description

In our production environment, we prefer to use LZ4 instead of Snappy as compression for better performance, which is also recommended by RocksDB .

I plan to introduce LZ4 except for now existing snappy compression for keyed state in full checkpoint and savepoints.

Issue Links

• is related to: FLINK-6931 Support custom compression formats for checkpoints (+Upgrade/Compatibility)
• relates to: FLINK-33819 Support setting CompressType in RocksDBStateBackend
• links to: FLINK PR-7515
• links to: GitHub Pull Request #7515
• links to: GitHub Pull Request #7819