[HADOOP-15822] zstd compressor can fail with a small output buffer

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.9.0, 3.0.0
• Fix Version/s: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2
• Target Version/s: 2.10.0, 3.2.0, 2.9.2, 3.0.4, 3.1.2
• Hadoop Flags: Reviewed
• Assignee: Jason Darrell Lowe
• Reporter: Jason Darrell Lowe
• Created: 05/Oct/18 19:46
• Updated: 24/Oct/18 05:12
• Resolved: 24/Oct/18 05:03

Description

TestZStandardCompressorDecompressor fails a couple of tests on my machine with the latest zstd library (1.3.5). Compression can fail to successfully finalize the stream when a small output buffer is used resulting in a failed to init error, and decompression with a direct buffer can fail with an invalid src size error.

Issue Links

• is caused by: HADOOP-13578 Add Codec for ZStandard Compression