[HDFS-9038] DFS reserved space is erroneously counted towards non-DFS used.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.1
• Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha2
• Component/s: datanode
• Hadoop Flags: Reviewed
• Assignee: Brahma Reddy Battula
• Reporter: Chris Nauroth
• Created: 08/Sep/15 22:28
• Updated: 10/May/17 09:38
• Resolved: 08/Sep/16 21:10

Description

HDFS-5215 changed the DataNode volume available space calculation to consider the reserved space held by the dfs.datanode.du.reserved configuration property. As a side effect, reserved space is now counted towards non-DFS used. I don't believe it was intentional to change the definition of non-DFS used. This issue proposes restoring the prior behavior: do not count reserved space towards non-DFS used.

Issue Links

• breaks: HDFS-11312 Fix incompatible tag number change for nonDfsUsed in DatanodeInfoProto
• is broken by: HDFS-5215 dfs.datanode.du.reserved is not considered while computing available space
• is related to: HDFS-8072 Reserved RBW space is not released if client terminates while writing block