[HDFS-8072] Reserved RBW space is not released if client terminates while writing block

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.7.0, 2.6.1, 3.0.0-alpha1
• Component/s: datanode
• Labels: 2.6.1-candidate
• Hadoop Flags: Reviewed
• Assignee: Arpit Agarwal
• Reporter: Arpit Agarwal
• Created: 07/Apr/15 02:31
• Updated: 30/Aug/16 01:37
• Resolved: 08/Apr/15 18:40

Description

The DataNode reserves space for a full block when creating an RBW block (introduced in HDFS-6898).

The reserved space is released incrementally as data is written to disk and fully when the block is finalized. However if the client process terminates unexpectedly mid-write then the reserved space is not released until the DN is restarted.

Issue Links

• duplicates: HDFS-8547 The space reservation for RBW block is leaking
• is broken by: HDFS-6898 DN must reserve space for a full block when an RBW block is created
• relates to: HDFS-9038 DFS reserved space is erroneously counted towards non-DFS used.