[HBASE-7253] Compaction Tool

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 0.95.2
• Fix Version/s: 0.95.0
• Component/s: Compaction

Release Note

The CompactionTool works at file-system level, so the table should be disabled.

The compaction process uses the same hbase-site.xml configuration property used by the server, like
"hbase.hstore.compactionThreshold" & co.

You can compact the whole table or just a single region or family,
and the input of the CompactionTool is a fs path.

You can run the compaction as a MapReduce Job, or as a local process.
Each family can be compacted in parallel if you use the -mapreduce option.

To compact "TestTable" family "cf1" of region "e450da04b1a10099b618bec031e0f951"
bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool hdfs:///hbase/TestTable/e450da04b1a10099b618bec031e0f951/cf1

To compact all the families of region "e450da04b1a10099b618bec031e0f951":
bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool hdfs:///hbase/TestTable/e450da04b1a10099b618bec031e0f951

To compact all regions and family of the Table:
bin/hbase org.apache.hadoop.hbase.regionserver.CompactionTool -mapred hdfs:///hbase/TestTable

• Assignee: Matteo Bertozzi
• Reporter: Matteo Bertozzi
• Created: 01/Dec/12 22:32
• Updated: 23/Sep/13 18:31
• Resolved: 03/Dec/12 20:35

Description

In HBASE-5616, as part of the compaction code refactor, a CompactionTool was added.

but there are some issues:

• The tool is under test/
• mockito is required, so the "test" scope should be removed from the pom.xml, otherwise the tool doesn't start
• The mock, used by the tool, is mocking HRegion.getRegionInfo() but some code (Store) uses HRegion.regionInfo directly HStore.java#L2021, HStore.java#L1389, HStore.java#L1402 and you end up with a NPE in the tool.
• The Mocked Store uses a dummy family and the compacted files doesn't get the same family properties specified (compression, encoding, ...)
• at the end of compaction CompactionTool.java#L155, on by default, the compaction file is removed (note that the compacted one are already removed inside the store.compact()... and you end up with an empty dir, if you compact everything.

I've fixed some stuff and added support to:

• Run the compaction as a MR Job
• Specify a Table (compact each region/family)
• Specify a Region (compact each family)
• Specify a Family (as before)

Sub-Tasks

• Backport Compaction Tool to 0.94: Closed