[HBASE-4618] HBase backups

• Type: Umbrella
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Component/s: documentation, regionserver
• Assignee: Unassigned
• Reporter: Karthik Ranganathan
• Created: 19/Oct/11 15:27
• Updated: 12/Jun/22 18:38
• Resolved: 12/Jun/22 18:35

Description

We have been working on the ability to do backups in HBase with different levels of protection. This is an umbrella task for all the backup related changes. Here are some kinds of changes - will create separate issues for them:

Roughly here are a few flavors of backups giving increasing levels of guarentees:
1. Per cf backups
2. Multi-cf backups with row atomicity preserved
3. Multi-cf backups with row atomicity and point in time recovery.

On the perf dimension, here is a list of improvements:
1. Copy the files - regular hadoop "cp"
2. Use fast copy - copy blocks and stitch them together, saves top of rack bandwidth
3. Use fast copy with hard links - no file copy, it does only ext3 level linking.

On the durability of data side:
1. Ability to backup data onto the same racks as those running HBase
2. Intra-datacenter backup
3. Inter datacenter backup

Restores:
1. Restore with a table name different from the backed up table name
2. Restore a backed up table wen HBase cluster is not running at restore time
3. Restore into a live and running cluster

Operationally:
1. How to setup backups in live cluster
2. Setting up intra-DC
3. cross-DC backups
4. Verifying a backup is good

Issue Links

• blocks: HBASE-4907 Port 89-fb changes to trunk
• is related to: HBASE-12342 HBase Backup Options
• requires: HBASE-4591 TTL for old HLogs should be calculated from last modification time.

Sub-Tasks

• Document architecture of backups: Closed
• Ability to export the list of files for a some or all column families for a given region: Closed
• Replay the required hlog edits to make the backup preserve row atomicity.: Closed
• MR based copier for copying HFiles: Closed
• Add the various scripts to take HBase backups: Closed
• Importer for exported tables: Closed
• Ability to verify backed up data: Closed
• List the steps required to enable HBase backups: Closed
• List HDFS enhancements to speed up backups for HBase: Closed
• MR based copier for copying HFiles (trunk version): Closed