[HBASE-23680] RegionProcedureStore missing cleaning of hfile archive

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.0
• Fix Version/s: 3.0.0-alpha-1, 2.3.0
• Component/s: proc-v2, RegionProcedureStore
• Hadoop Flags: Reviewed

Release Note

Add a new config to hbase-default.xml

  <property>
    <name>hbase.procedure.store.region.hfilecleaner.plugins</name>
    <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner</value>
    <description>A comma-separated list of BaseHFileCleanerDelegate invoked by
    the RegionProcedureStore HFileCleaner service. These HFiles cleaners are
    called in order, so put the cleaner that prunes the most files in front. To
    implement your own BaseHFileCleanerDelegate, just put it in HBase's classpath
    and add the fully qualified class name here. Always add the above
    default hfile cleaners in the list as they will be overwritten in
    hbase-site.xml.</description>
  </property>

It will share the same TTL with other HFileCleaners. And you can also implement your own cleaner and change this property to enable it.

• Assignee: Duo Zhang
• Reporter: Michael Stack
• Created: 11/Jan/20 00:13
• Updated: 19/Jan/20 14:07
• Resolved: 18/Jan/20 12:59

Description

See tail of parent issue. The new RegionProcedureStore accumulates deleted hfiles in its local archive dir. Needs a cleaner like the one that watches over /hbase/archive.

Is there a problem clearning the new $masterproc$ files from the oldWALs too? These seem to stick around also.

Issue Links

• links to: GitHub Pull Request #1022