The index-compation and data-compaction variants of CompactingMemStore are introduced. In both types the active (mutable) segment is periodically flushed-in-memory and is added as immutable segment in the compaction pipeline. The CompactingMemStore of index-compaction type is merging all immutable segments of the compacting pipeline into one. The merging of N segments is explained below. The CompactingMemStore of data-compaction type is compacting all immutable segments of the compacting pipeline into one. After the merge/compaction the old segments in the compacting pipeline are replaced with one new.
Before explaining the process of merging N old segments into new one, note that segment structure includes ordered index that allows traversing the cells data efficiently. The merge is copying the ordered indexes of the old segments into one ordered index of new segment. No data is copied, no cells are filtered. Alternatively, in the process of compacting N old segments into new one, both data and index are copied. The old cells are filtered, meaning upon compaction unused versions of the cells are not copied so the new segment has less data then all old ones.
This issue introduces only the merging ability and simplifies the user intervention for switching between types. The previous CompactingMemStore structure was added by
HBASE-16420 and
HBASE-16421. The future refinements of the policy or merging/compacting will come in
HBASE-16417.
In order to create a table with CompactingMemStore as a MemStore one should use:
create ‘<tablename>’, {NAME => ‘<cfname>’, IN_MEMORY_COMPACTION => true}
IN_MEMORY_COMPACTION default is false, so table created as following will have the known DefaultMemStore as a MemStore.
create ‘<tablename>’, {NAME => ‘<cfname>’}
The default type of CompactingMemStore is index-compaction. In order to change it to data-compaction one should add to the hbase-site.xml
<property>
<name>hbase.hregion.compacting.memstore.type</name>
<value>data-compaction</value>
</property>
in addition to creating the table as following
create ‘<tablename>’, {NAME => ‘<cfname>’, IN_MEMORY_COMPACTION => true}
Sub-task
Major
Anastasia Braginsky