[HADOOP-9152] HDFS can report negative DFS Used on clusters with very small amounts of data

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.2-alpha
• Fix Version/s: 2.0.3-alpha, 0.23.6
• Component/s: fs
• Target Version/s: 2.0.3-alpha
• Hadoop Flags: Reviewed
• Assignee: Brock Noland
• Reporter: Brock Noland
• Created: 26/Nov/12 22:46
• Updated: 15/Feb/13 13:11
• Resolved: 18/Dec/12 19:51

Description

I had a near empty HDFS instance where I was creating a file and deleting it very quickly. I noticed that HDFS sometimes reported a negative DFS used.

root@brock0-1 ~]# sudo -u hdfs -i hdfs dfsadmin -report
Configured Capacity: 97233235968 (90.56 GB)
Present Capacity: 84289609707 (78.5 GB)
DFS Remaining: 84426645504 (78.63 GB)
DFS Used: -137035797 (-133824.02 KB)
DFS Used%: -0.16%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Live datanodes:
Name: 127.0.0.1:50010 (localhost)
Hostname: brock0-1.ent.cloudera.com
Decommission Status : Normal
Configured Capacity: 97233235968 (90.56 GB)
DFS Used: -137035797 (-133824.02 KB)
Non DFS Used: 12943626261 (12.05 GB)
DFS Remaining: 84426645504 (78.63 GB)
DFS Used%: -0.14%
DFS Remaining%: 86.83%
Last contact: Thu Nov 22 18:25:37 PST 2012




[root@brock0-1 ~]# sudo -u hdfs -i hdfs dfsadmin -report
Configured Capacity: 97233235968 (90.56 GB)
Present Capacity: 84426973184 (78.63 GB)
DFS Remaining: 84426629120 (78.63 GB)
DFS Used: 344064 (336 KB)
DFS Used%: 0%
Under replicated blocks: 0
Blocks with corrupt replicas: 0
Missing blocks: 0

-------------------------------------------------
Datanodes available: 1 (1 total, 0 dead)

Live datanodes:
Name: 127.0.0.1:50010 (localhost)
Hostname: brock0-1.ent.cloudera.com
Decommission Status : Normal
Configured Capacity: 97233235968 (90.56 GB)
DFS Used: 344064 (336 KB)
Non DFS Used: 12806262784 (11.93 GB)
DFS Remaining: 84426629120 (78.63 GB)
DFS Used%: 0%
DFS Remaining%: 86.83%
Last contact: Thu Nov 22 18:28:47 PST 2012