[HDFS-6651] Deletion failure can leak inodes permanently

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Target Version/s: 2.5.0
• Hadoop Flags: Incompatible change, Reviewed
• Assignee: Jing Zhao
• Reporter: Kihwal Lee
• Created: 09/Jul/14 17:39
• Updated: 05/Oct/15 12:10
• Resolved: 03/Feb/15 00:37

Description

As discussed in HDFS-6618, if a deletion of tree fails in the middle, any collected inodes and blocks will not be removed from INodeMap and BlocksMap.

Since fsimage is saved by iterating over INodeMap, the leak will persist across name node restart. Although blanked out inodes will not have reference to blocks, blocks will still refer to the inode as BlockCollection. As long as it is not null, blocks will live on. The leaked blocks from blanked out inodes will go away after restart.

Options (when delete fails in the middle)

• Complete the partial delete: edit log the partial delete and remove inodes and blocks.
• Somehow undo the partial delete.
• Check quota for snapshot diff beforehand for the whole subtree.
• Ignore quota check during delete even if snapshot is present.

Issue Links

• is duplicated by: HDFS-9194 AlreadyBeingCreatedException ... because pendingCreates is non-null but no leases found.