[HADOOP-17825] Add BuiltInGzipCompressor

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0
• Component/s: common
• Labels: pull-request-available
• Hadoop Flags: Reviewed
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Created: 31/Jul/21 03:12
• Updated: 09/Sep/21 04:26
• Resolved: 16/Aug/21 17:09

Description

Currently, GzipCodec only supports BuiltInGzipDecompressor, if native zlib is not loaded. So, without Hadoop native codec installed, saving SequenceFile using GzipCodec will throw exception like "SequenceFile doesn't work with GzipCodec without native-hadoop code!"

Same as other codecs which we migrated to using prepared packages (lz4, snappy), it will be better if we support GzipCodec generally without Hadoop native codec installed. Similar to BuiltInGzipDecompressor, we can use Java Deflater to support BuiltInGzipCompressor.

Issue Links

• causes: HADOOP-17877 BuiltInGzipCompressor header and trailer should not be static variables
• relates to: HADOOP-17887 Remove GzipOutputStream
• relates to: HADOOP-17868 Add more test for the BuiltInGzipCompressor
• links to: GitHub Pull Request #3250