Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11915

Sync rbw dir on the first hsync() to avoid file lost on power failure

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Critical
    • Resolution: Fixed
    • None
    • 3.1.0, 2.10.0, 2.9.1, 3.0.1
    • None
    • None

    Description

      As discussed in HDFS-5042, there is a chance to lose blocks on power failure if rbw file creation entry is not yet sync to device. Then the block created is nowhere exists on disk. Neither in rbw nor in finalized.

      As suggested by kihwal, will discuss and track it in this JIRA.

      As suggested by vinayrpet, May be first hsync() request on block file can call fsync on its parent directory (rbw) directory.

      Attachments

        1. HDFS-11915-01.patch
          2 kB
          Vinayakumar B
        2. HDFS-11915-branch-2-01.patch
          2 kB
          Vinayakumar B
        3. HDFS-11915-branch-2-02.patch
          5 kB
          Vinayakumar B

        Issue Links

          Activity

            People

              vinayakumarb Vinayakumar B
              kanaka Kanaka Kumar Avvaru
              Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: