[HBASE-22648] Snapshot TTL

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0-alpha-1
• Fix Version/s: 3.0.0-alpha-1, 1.5.0, 2.3.0
• Component/s: snapshots
• Hadoop Flags: Reviewed

Release Note

Feature: Take a Snapshot With TTL for auto-cleanup

Attribute:
1. TTL
     - Specify TTL in sec while creating snapshot. e.g. snapshot 'mytable', 'snapshot1234', {TTL => 86400} (snapshot to be auto-cleaned after 24 hr)

Configs:
1. Default Snapshot TTL:
     - FOREVER by default
     - User specified Default TTL(sec) with config: hbase.master.snapshot.ttl

2. If Snapshot cleanup is supposed to be stopped due to some snapshot restore activity, disable it with config:
     - hbase.master.cleaner.snapshot.disable: "true"
    With this config, HMaster needs restart just like any other hbase-site config.


For more details, see the section "Take a Snapshot With TTL" in the HBase Reference Guide.

• Assignee: Viraj Jasani
• Reporter: Andrew Kyle Purtell
• Created: 02/Jul/19 09:47
• Updated: 21/Jul/24 15:43
• Resolved: 23/Jul/19 16:54

Description

Snapshots have a lifecycle that is independent from the table from which they are created. Although data in a table may be stored with TTL the data files containing them become frozen by the snapshot. Space consumed by expired cells will not be reclaimed by normal table housekeeping like compaction. While this is expected it can be inconvenient at scale. When many snapshots are under management and the data in various tables is expired by TTL some notion of optional TTL (and optional default TTL) for snapshots could be useful. It will help prevent the accumulation of junk files by automatically dropping the snapshot after the assigned TTL, making their data files eligible for cleaning. More comprehensive snapshot lifecycle management may be considered in the future but this one case is expected to be immediately useful given TTls on data are commonly applied for similar convenience. 

Issue Links

• is related to: HBASE-27671 Client should not be able to restore/clone a snapshot after it's TTL has expired
• relates to: HBASE-28734 Improve HBase shell snapshot command Doc with TTL option
• relates to: HBASE-24611 Bring back old constructor of SnapshotDescription
• links to: GitHub Pull Request #371