[HDFS-11915] Sync rbw dir on the first hsync() to avoid file lost on power failure

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.1.0, 2.10.0, 2.9.1, 3.0.1
• Assignee: Vinayakumar B
• Reporter: Kanaka Kumar Avvaru
• Created: 02/Jun/17 08:02
• Updated: 17/May/18 21:05
• Resolved: 12/Jan/18 18:02

Description

As discussed in HDFS-5042, there is a chance to lose blocks on power failure if rbw file creation entry is not yet sync to device. Then the block created is nowhere exists on disk. Neither in rbw nor in finalized.

As suggested by kihwal, will discuss and track it in this JIRA.

As suggested by vinayrpet, May be first hsync() request on block file can call fsync on its parent directory (rbw) directory.

Issue Links

• breaks: HDFS-13586 Fsync fails on directories on Windows
• is related to: HDFS-5042 Completed files lost after power failure
• relates to: HDFS-13487 Backport HDFS-11915 to branch-2.8, branch-2.7