Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-2975

Rename with overwrite flag true can make NameNode to stuck in safemode on NN (crash + restart).

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: namenode
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When we rename the file with overwrite flag as true, it will delete the destination file blocks. After deleting the blocks, whenever it releases the fsNameSystem lock, NN can give the invalidation work to corresponding DNs to delete the blocks.
      Parallaly it will sync the rename related edits to editlog file. At this step before NN sync the edits if NN crashes, NN can stuck into safemode on restart. This is because block already deleted from the DN as part of invalidations. But dst file still exist as rename edits not persisted in log file and no DN will report that blocks now.

      This is similar to HDFS-2815

        Attachments

        1. HDFS-2975.001.patch
          7 kB
          Yi Liu
        2. HDFS-2975.002.patch
          8 kB
          Yi Liu

          Activity

            People

            • Assignee:
              hitliuyi Yi Liu
              Reporter:
              umamaheswararao Uma Maheswara Rao G
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: