[SPARK-33263] Configurable StateStore compression codec

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL, Structured Streaming
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 27/Oct/20 21:00
• Updated: 29/Oct/20 14:45
• Resolved: 29/Oct/20 14:45

Description

Currently the compression codec of StateStore is not configurable and hard-coded to be lz4. It is better if we can follow Spark other modules to configure the compression codec of StateStore.

Issue Links

• links to: [Github] Pull Request #30162 (viirya)