Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6820

Namenode fails to boot if the file system reorders rename operations

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.2.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      After studying the steps HDFS name node takes to update the logs we found the following bug. The bug may not manifest in all current file system implementations, but it is possible in file systems that reorder metadata operations. e.g. btrfs

      Looking at the strace of HDFS name node we see the following when updating the image:
      create(fsimage.chk)
      append(fsimage.chk)
      fsync(fsimage.chk)

      create(fsimage.md5.tmp)
      append(fsimage.md5.tmp)
      fsync(fsimage.md5.tmp)

      rename(fsimage.md5, fsimage.md5.tmp)
      rename(fsimage, fsimage.chk)

      If the file system reorders the last two rename operations and the system crashes before the second rename is persisted on disk, the system may end up with a fsimage that does not have a corresponding md5 file. In this case HDFS namenode does not boot.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              samera Samer Al-Kiswany

              Dates

              • Created:
                Updated:

                Issue deployment