Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6651

Deletion failure can leak inodes permanently

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 2.7.0
    • None
    • None
    • Incompatible change, Reviewed

    Description

      As discussed in HDFS-6618, if a deletion of tree fails in the middle, any collected inodes and blocks will not be removed from INodeMap and BlocksMap.

      Since fsimage is saved by iterating over INodeMap, the leak will persist across name node restart. Although blanked out inodes will not have reference to blocks, blocks will still refer to the inode as BlockCollection. As long as it is not null, blocks will live on. The leaked blocks from blanked out inodes will go away after restart.

      Options (when delete fails in the middle)

      • Complete the partial delete: edit log the partial delete and remove inodes and blocks.
      • Somehow undo the partial delete.
      • Check quota for snapshot diff beforehand for the whole subtree.
      • Ignore quota check during delete even if snapshot is present.

      Attachments

        1. HDFS-6651.000.patch
          77 kB
          Jing Zhao
        2. HDFS-6651.001.patch
          80 kB
          Jing Zhao
        3. HDFS-6651.002.patch
          82 kB
          Jing Zhao

        Issue Links

          Activity

            People

              jingzhao Jing Zhao
              kihwal Kihwal Lee
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: