[HBASE-3166] HBase exporter should compress output files by default (or at least allow this as an option)

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Duplicate
• Affects Version/s: 0.20.6
• Component/s: util
• Assignee: Unassigned
• Reporter: Josh Rosenblum
• Created: 28/Oct/10 21:37
• Updated: 12/Jun/22 00:32
• Resolved: 28/Oct/10 21:57

Description

The HBase exporter puts (key, Result) pairs as keys and values into an output sequence file.

There could be significant savings at low cost if at least default compression was enabled on this output sequence file.

In createSubmittableJob(), this might be as simple as adding the following:

SequenceFileOutputFormat.setOutputCompressionType(job, SequenceFile.CompressionType.BLOCK);
SequenceFileOutputFormat.setCompressOutput(job, true);
FileOutputFormat.setOutputCompressorClass(job, DefaultCodec.class);

Issue Links

• duplicates: HBASE-2225 Enable compression in HBase Export