[SPARK-19958] Support ZStandard Compression

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.1.0
• Component/s: Spark Core
• Assignee: Unassigned
• Reporter: Dongjin Lee
• Created: 15/Mar/17 08:24
• Updated: 15/Mar/17 09:02
• Resolved: 15/Mar/17 08:32

Description

Hadoop[^1] & HBase[^2] started to support ZStandard Compression from their recent releases. Supporting this compression codec also requires adding a new configuration for default compression level, for example, 'spark.io.compression.zstandard.level.'

[^1]: https://issues.apache.org/jira/browse/HADOOP-13578
[^2]: https://issues.apache.org/jira/browse/HBASE-16710

Issue Links

• duplicates: SPARK-19112 add codec for ZStandard