[HDFS-15477] Enforce ordered snapshot deletion

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: snapshots
• Assignee: Tsz-wo Sze
• Reporter: Tsz-wo Sze
• Created: 18/Jul/20 19:04
• Updated: 14/Aug/20 17:37

Description

Snapshot deletion has caused a few bugs earlier such as HDFS-13101 and HDFS-15313. In this JIRA, we propose a configurable feature for enforcing ordered snapshot deletion – only the earliest snapshot is actually deleted from the file system. The other snapshots are only marked as deleted. They will be actually deleted from the file system until all the earlier snapshots are deleted.

The reason of enforcing ordered snapshot deletion is based on the observation that the logic of deleting the earliest snapshot is much simpler since the prior snapshot does not exist. All the previous bugs are caused by removing inodes from the prior snapshot.

One drawback of ordered snapshot deletion is that the resources are not yet released for deleting the non-earliest snapshots since they are only marked as deleted but not actually deleted.

Issue Links

• is related to: HDFS-15516 Add info for create flags in NameNode audit logs
• is related to: HDFS-15513 Allow client to query snapshot status on one directory
• is related to: HDFS-15519 Check inaccessible INodes in FsImageValidation

Sub-Tasks

• Metric for ordered snapshot deletion GC thread: Open
• WebHDFS: Support provisionSnapshotTrash: Open
• Update Apache documentation for new ordered snapshot deletion feature: Open