Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2975

Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart).

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 2.6.0
    • namenode
    • None
    • Reviewed

    Description

      When we rename the file with overwrite flag as true, it will delete the destination file blocks. After deleting the blocks, whenever it releases the fsNameSystem lock, NN can give the invalidation work to corresponding DNs to delete the blocks.
      Parallaly it will sync the rename related edits to editlog file. At this step before NN sync the edits if NN crashes, NN can stuck into safemode on restart. This is because block already deleted from the DN as part of invalidations. But dst file still exist as rename edits not persisted in log file and no DN will report that blocks now.

      This is similar to HDFS-2815

      Attachments

        1. HDFS-2975.002.patch
          8 kB
          Yi Liu
        2. HDFS-2975.001.patch
          7 kB
          Yi Liu

        Activity

          People

            hitliuyi Yi Liu
            umamaheswararao Uma Maheswara Rao G
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: