[HBASE-25756] Support alternate compression for major and minor compactions

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha-1, 2.5.0
• Component/s: Compaction
• Hadoop Flags: Reviewed

Release Note

It is now possible to specify alternate compression algorithms for major or minor compactions, via new ColumnFamilyBuilder or HColumnDescriptor methods {get,set}{Major,Minor}CompressionType(), or shell schema attributes COMPRESSION_COMPACT_MAJOR or COMPRESSION_COMPACT_MINOR. This can be used to select a fast algorithm for frequent minor compactions and a slower algorithm offering better compression ratios for infrequent major compactions.

• Assignee: Andrew Kyle Purtell
• Reporter: Andrew Kyle Purtell
• Created: 09/Apr/21 19:34
• Updated: 24/Apr/21 18:08
• Resolved: 23/Apr/21 23:20

Description

Resurrecting HBASE-2988.

Currently we can specify a compression algorithm for a column family, or, optionally an alternative compression algorithm to use when compacting (as opposed to flushing). Extend this to optionally specify alternative compression algorithms for minor or major compactions. The interest here is specifically a separate alternative algorithm for major compactions, which might be an expensive but aggressively compressing algorithm like bzip2 or lzma (with custom patches).

Issue Links

• is related to: HBASE-2988 Support alternate compression for major compactions
• links to: GitHub Pull Request #3142