[HADOOP-13578] Add Codec for ZStandard Compression

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Target Version/s: 3.0.0-alpha2
• Hadoop Flags: Reviewed
• Assignee: churro morales
• Reporter: churro morales
• Created: 02/Sep/16 23:22
• Updated: 01/Jun/21 12:38
• Resolved: 04/Jan/17 16:22

Description

ZStandard: https://github.com/facebook/zstd has been used in production for 6 months by facebook now. v1.0 was recently released. Create a codec for this library.

Issue Links

• causes: HADOOP-15820 ZStandardDecompressor native code sets an integer field as a long
• causes: HADOOP-15859 ZStandardDecompressor.c mistakes a class for an instance
• causes: HADOOP-15822 zstd compressor can fail with a small output buffer
• causes: HADOOP-17096 Fix ZStandardCompressor input buffer offset
• is depended upon by: HADOOP-14482 Update BUILDING.txt to include the correct steps to install zstd library
• is related to: HADOOP-17219 ZStandardCodec compression mail fail(generic error) when encounter specific file
• is related to: HADOOP-14805 Upgrade to zstd 1.3.1
• is related to: YETUS-895 Add ZStandard Compression support in Hadoop precommit
• relates to: HADOOP-16359 Bundle ZSTD native in branch-2
• relates to: HBASE-16710 Add ZStandard Codec to Compression.java
• mentioned in: Page Loading...