[KAFKA-4514] Add Codec for ZStandard Compression

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: compression
• Assignee: Dongjin Lee
• Reporter: Thomas Graves
• Created: 08/Dec/16 18:23
• Updated: 10/Oct/18 22:16
• Resolved: 10/Oct/18 00:16

Description

ZStandard: https://github.com/facebook/zstd and http://facebook.github.io/zstd/ has been in use for a while now. v1.0 was recently released. Hadoop (https://issues.apache.org/jira/browse/HADOOP-13578) and others are adopting it.

We have done some initial trials and seen good results. Zstd seems to give great results => Gzip level Compression with Lz4 level CPU.

Issue Links

• links to: GitHub Pull Request #2267
• links to: GitHub Pull Request #2267
• links to: GitHub Pull Request #5777
• mentioned in: Page Loading...