Details
-
Wish
-
Status: Resolved
-
Major
-
Resolution: Resolved
-
1.7.1, 3.0.0-alpha-2, 2.4.10
-
None
Description
Currently, region will not split when the number of store files is up to the configed blocking count, by `hbase.hstore.blockingStoreFiles`.
The relevant codes are as follows,
the CompactSplit#requestSplit() (called by the MemstoreFlusher and CompactionRunner) checks the compaction priority of the region, if the compact priority < PRIORITY_USER, the region will not split.
public synchronized boolean requestSplit(final Region r) { // don't split regions that are blocking HRegion hr = (HRegion)r; try { if (shouldSplitRegion() && hr.getCompactPriority() >= PRIORITY_USER) { byte[] midKey = hr.checkSplit().orElse(null); if (midKey != null) { requestSplit(r, midKey); return true; } } ....
But the region's compact priority is the minimum of all the stores, when the number of storefiles in a store is larger than the configed `hbase.hstore.blockingStoreFiles`, the compact priority will be a negative number, while PRIORITY_USER = 1.
public int getStoreCompactionPriority() { int priority = blockingFileCount - storefiles.size(); return (priority == HStore.PRIORITY_USER) ? priority + 1 : priority; }
As a result, when a region size is up to the split limit, but its speed of reducing the number of files through compaction is slower than the speed of generating new files(e.g. compacting L0 files to stripes, bulk load, flush memstore), the region will never split.
The problem is obvious in StripeStoreEngine, though memstore flushing is pending when store file count up to the blocking count, each L0 compaction may generate the stripe count new files to each stripe. And in this scenario, since the store always compact priority to split, the stripe count is larger and larger, the new files generated by compact is more and more, no split in the end. While split can divide the compaction pressure(1 parent compaction + 2 children compaction can be reduced to 2 children compaction).
We can add a configuration to enable split when blocking, not only keep the origin behavior but also support flexible control.