[HBASE-26342] Support custom paths of independent configuration and pool for hfile cleaner

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.7.1, 3.0.0-alpha-2, 2.4.10
• Fix Version/s: 2.5.0, 3.0.0-alpha-3
• Component/s: master

Release Note

Configure the custom hifle paths (under archive directory), e.g. data/default/testTable1,data/default/testTable2 by "hbase.master.hfile.cleaner.custom.paths".
Configure hfile cleaner classes for the custom paths by "hbase.master.hfilecleaner.custom.paths.plugins".
Configure the shared pool size of custom hfile cleaner paths by "hbase.cleaner.custom.hfiles.pool.size".

• Assignee: Xiaolin Ha
• Reporter: Xiaolin Ha
• Created: 09/Oct/21 05:34
• Updated: 01/Sep/22 10:57
• Resolved: 24/May/22 08:03

Description

With this, we can clean some paths more quickly.

We found in our cluster, when the very huge table with thousands of regions and high write throughputs and many snapshots tables on the same cluster, the speed of delete files in  archive path will lower than the speed of moved in files by compaction. Then archive may remains PB level data. 

The bottleneck is in cleaner but not in the thread pool size or queue size. It is because there is synchronized lock in SnapshotFileCache, and a batch of files need once SnapshotFileCache#refreshCache(), which look through all the snapshot dirs.

The speed of clear a path without the SnapshotHFileCleaner is thirty times faster.

Issue Links

• links to: GitHub Pull Request #4403
• links to: GitHub Pull Request #4461