Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
Current migration has no support for compression. You can always enable compression after migration and it should take effect after the next major compaction. For anyone else who may still have a significant amount of compressed data in 0.19, this might prove useful.
We have a large amount of data stored in 0.19 under GZip compression, at an effective 10x rate. Out of concern for migration speed and the risk of using more capacity than our cluster has, we didn't want to first write out all the data uncompressed before enabling LZO compression.
I'm attaching a small patch that allowed us to migrate the data directly to a compressed table format. To use it, add a migrate.compression property to hbase-site.xml with a value of LZO or GZ, and it should compress all tables during migration.