[HBASE-14142] HBase Backup/Restore Phase 3: Edits deduplication during backup

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Invalid
• Fix Version/s: HBASE-7912
• Assignee: Unassigned
• Reporter: Vladimir Rodionov
• Created: 21/Jul/15 21:20
• Updated: 01/Jul/22 20:09
• Resolved: 08/Jun/16 19:01

Description

As since we do not record last backed up sequence ids (MVCC) and do not restore up to that sequence id - that is kind of tricky, there will be some duplicates of KVs in store files after first incremental restore after full backup. These duplicates are result of how we do full backup and first incremental backup after full one. During full backup we perform distributed log roll and record, for every RS, last WAL timestamp, then we do snapshot. The next WAL after recorded one will make it into a next incremental backup set, but it will contains some edits (puts, deletes) which have been recorded by a previous snapshot. During restore, we, first, restore snapshot, then we will re-play WALs and this operation can create some duplicates of KVs in different store files.

Issue Links

• is part of: HBASE-14414 HBase Backup/Restore Phase 3