[HBASE-5547] Don't delete HFiles when in "backup mode"

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.95.0
• Hadoop Flags: Reviewed

Release Note

All HFiles are now automatically archived to the configured "hbase.table.archive.directory". HFiles are cleaned similarly to HLogs - a cleaner delegate chain that is instantiated as per the configured classes. Similar to hlog cleaners, a TimeToLiveHFileCleaner is used by default, and specified under the "hbase.master.hfilecleaner.plugins" configuration key.

In unifying the two cleaner interfaces, the new configuration parameter to use the TimeToLiveHLogCleaner is:
  <property>
    <name>hbase.master.logcleaner.plugins</name>
    <value>org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner</value>
 </property>

It is still enabled by default, so nothing needs to be changed if you are not modifying the logcleaners.

• Assignee: Jesse Yates
• Reporter: Lars Hofhansl
• Created: 09/Mar/12 05:15
• Updated: 28/Jan/22 00:58
• Resolved: 15/Sep/12 04:21

Description

This came up in a discussion I had with Stack.
It would be nice if HBase could be notified that a backup is in progress (via a znode for example) and in that case either:
1. rename HFiles to be delete to <file>.bck
2. rename the HFiles into a special directory
3. rename them to a general trash directory (which would not need to be tied to backup mode).

That way it should be able to get a consistent backup based on HFiles (HDFS snapshots or hard links would be better options here, but we do not have those).

#1 makes cleanup a bit harder.

Issue Links

• blocks: HBASE-6055 Offline Snapshots in HBase 0.96
• is related to: HBASE-6796 Backport HBASE-5547, Don't delete HFiles in backup mode.
• is related to: HBASE-50 Snapshot of table
• is required by: HBASE-26718 HFileArchiver can remove referenced StoreFiles from the archive
• relates to: HBASE-6441 MasterFS doesn't set scheme for internal FileSystem
• relates to: HBASE-6055 Offline Snapshots in HBase 0.96
• relates to: HBASE-6205 Support an option to keep data of dropped table for some time

Sub-Tasks

• Add test for archiving/cleaning across multiple tables where some tables are retained and others aren't: Closed