Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6651

Deletion failure can leak inodes permanently

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      As discussed in HDFS-6618, if a deletion of tree fails in the middle, any collected inodes and blocks will not be removed from INodeMap and BlocksMap.

      Since fsimage is saved by iterating over INodeMap, the leak will persist across name node restart. Although blanked out inodes will not have reference to blocks, blocks will still refer to the inode as BlockCollection. As long as it is not null, blocks will live on. The leaked blocks from blanked out inodes will go away after restart.

      Options (when delete fails in the middle)

      • Complete the partial delete: edit log the partial delete and remove inodes and blocks.
      • Somehow undo the partial delete.
      • Check quota for snapshot diff beforehand for the whole subtree.
      • Ignore quota check during delete even if snapshot is present.

        Attachments

        1. HDFS-6651.000.patch
          77 kB
          Jing Zhao
        2. HDFS-6651.001.patch
          80 kB
          Jing Zhao
        3. HDFS-6651.002.patch
          82 kB
          Jing Zhao

          Issue Links

            Activity

              People

              • Assignee:
                jingzhao Jing Zhao
                Reporter:
                kihwal Kihwal Lee
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: