[HDFS-7268] Can't append with SYNC_BLOCK

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.4.1
• Component/s: hdfs-client
• Assignee: Unassigned
• Reporter: Bogdan Raducanu
• Created: 20/Oct/14 16:32
• Updated: 06/Jan/17 22:03

Description

It seems to me that currently it's not possible to start appending to a file with CreateFlag.SYNC_BLOCK behavior (i.e. hsync after each block). So, I think, the only way to guarantee durability when appending is if the user calls hsync at the end of each block.

FileSystem.append doesn't accept a CreateFlag argument.
FileSystem.create, on the other hand, ignores CreateFlag.APPEND afaics. I think the plan in HDFS-744 was to use this method if durability is needed.

It seems it might work through FileContext but in the end DFSOutputStream never sets shouldSyncBlock when appending anyway.
Or am I missing something?