[HBASE-25392] Direct insert compacted HFiles into data directory.

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed

Release Note

Introduces the DirectStoreCompactor, an extension of DefaultCompactor, which writes compactions result files straight into the store directory, rather than the temp dir approach implemented by DefaultCompactor.

In commitCompaction, it does not perform any renames. Should always be used together with PersistedStoreEngine proposed in HBASE-25395 and DirectStoreFlusher proposed in HBASE-25391, as it's up for StoreEngine implementations keep tracking of valid store files.

To enable this, the following properties must be set in RS configuration:

<property>
<name>hbase.storefile.tracking.persist.enabled</name><value>true</value>
</property>
<property>
<name>hbase.hstore.engine.class</name>
<value>org.apache.hadoop.hbase.regionserver.PersistedStoreEngine</value>
</property>
<property>
<name>hbase.regionserver.store.flush.context.class</name><value>org.apache.hadoop.hbase.regionserver.DirectStoreFlushContext</value>
</property>
<property>
<name>hbase.hstore.defaultengine.storeflusher.class</name><value>org.apache.hadoop.hbase.regionserver.DirectStoreFlusher</value></property>
<property>
<name>hbase.hstore.defaultengine.compactor.class</name>
<value>org.apache.hadoop.hbase.regionserver.compactions.DirectStoreCompactor</value>
</property>

• Assignee: Wellington Chevreuil
• Reporter: Tak-Lon (Stephen) Wu
• Created: 14/Dec/20 17:24
• Updated: 23/Jun/22 20:28
• Resolved: 28/Jul/21 18:36

Description

when performing compaction, write directly to the data directory

Issue Links

• is required by: HBASE-25396 Integration with direct insert into data/cf directory
• links to: GitHub Pull Request #3389