[HBASE-3474] HFileOutputFormat to use column family's compression algorithm

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.92.0
• Fix Version/s: 0.92.0
• Component/s: mapreduce

Environment

All

• Hadoop Flags: Reviewed
• Release Note: HFileOutputFormat to use column family's compression algorithm instead of a blanket all column family wide
• Assignee: Unassigned
• Reporter: Ashish Shinde
• Created: 25/Jan/11 04:46
• Updated: 20/Nov/15 12:43
• Resolved: 24/Mar/11 23:00

Description

HFileOutputFormat currently creates HFile writer's using a compression algorithm set as configuration "hbase.hregion.max.filesize" with default as no compression. The code does not take into account the compression algorithm configured for the table's column family. As a result bulk uploaded tables are not compressed until a major compaction is run on them. This could be fixed by using the column family descriptors while creating HFile writers.