[HBASE-14141] HBase Backup/Restore Phase 3: Filter WALs on backup to include only edits from backed up tables

• Type: New Feature
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 2.0.0
• Labels: backup
• Assignee: Vladimir Rodionov
• Reporter: Vladimir Rodionov
• Created: 21/Jul/15 21:17
• Updated: 01/Jul/22 20:08
• Resolved: 05/Apr/17 18:01

Description

High level design overview

• When incremental backup request comes for tables {t} we select all the tables already registered in a backup system - {T} and union them with {t}

  , which results in a new table set - U(t, T)

• For every table K from U(t,T) we perform the following:
  • Convert new WAL files into HFile applying table filter K (only edits for table T will pass the filter)
  • Move these HFile(s) to backup destination

During restore (incremental):

• We run full restore first
• Then collect all HFiles from intermediate incremental images and run them through HFileSplitterJob, which splits files into a current tables region boundaries
• Load these files using LoadIncrementalHFiles tool

Issue Links

• is part of: HBASE-14414 HBase Backup/Restore Phase 3