Hadoop Common
  1. Hadoop Common
  2. HADOOP-9152

HDFS can report negative DFS Used on clusters with very small amounts of data

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 2.0.2-alpha
    • Fix Version/s: 2.0.3-alpha, 0.23.6
    • Component/s: fs
    • Labels:
      None

      Description

      I had a near empty HDFS instance where I was creating a file and deleting it very quickly. I noticed that HDFS sometimes reported a negative DFS used.

      root@brock0-1 ~]# sudo -u hdfs -i hdfs dfsadmin -report
      Configured Capacity: 97233235968 (90.56 GB)
      Present Capacity: 84289609707 (78.5 GB)
      DFS Remaining: 84426645504 (78.63 GB)
      DFS Used: -137035797 (-133824.02 KB)
      DFS Used%: -0.16%
      Under replicated blocks: 0
      Blocks with corrupt replicas: 0
      Missing blocks: 0
      
      -------------------------------------------------
      Datanodes available: 1 (1 total, 0 dead)
      
      Live datanodes:
      Name: 127.0.0.1:50010 (localhost)
      Hostname: brock0-1.ent.cloudera.com
      Decommission Status : Normal
      Configured Capacity: 97233235968 (90.56 GB)
      DFS Used: -137035797 (-133824.02 KB)
      Non DFS Used: 12943626261 (12.05 GB)
      DFS Remaining: 84426645504 (78.63 GB)
      DFS Used%: -0.14%
      DFS Remaining%: 86.83%
      Last contact: Thu Nov 22 18:25:37 PST 2012
      
      
      
      
      [root@brock0-1 ~]# sudo -u hdfs -i hdfs dfsadmin -report
      Configured Capacity: 97233235968 (90.56 GB)
      Present Capacity: 84426973184 (78.63 GB)
      DFS Remaining: 84426629120 (78.63 GB)
      DFS Used: 344064 (336 KB)
      DFS Used%: 0%
      Under replicated blocks: 0
      Blocks with corrupt replicas: 0
      Missing blocks: 0
      
      -------------------------------------------------
      Datanodes available: 1 (1 total, 0 dead)
      
      Live datanodes:
      Name: 127.0.0.1:50010 (localhost)
      Hostname: brock0-1.ent.cloudera.com
      Decommission Status : Normal
      Configured Capacity: 97233235968 (90.56 GB)
      DFS Used: 344064 (336 KB)
      Non DFS Used: 12806262784 (11.93 GB)
      DFS Remaining: 84426629120 (78.63 GB)
      DFS Used%: 0%
      DFS Remaining%: 86.83%
      Last contact: Thu Nov 22 18:28:47 PST 2012
      

        Activity

        Arun C Murthy made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Thomas Graves made changes -
        Fix Version/s 0.23.6 [ 12323504 ]
        Aaron T. Myers made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.0.3-alpha [ 12323273 ]
        Resolution Fixed [ 1 ]
        Aaron T. Myers made changes -
        Target Version/s 2.0.3-alpha [ 12323273 ]
        Aaron T. Myers made changes -
        Project Hadoop HDFS [ 12310942 ] Hadoop Common [ 12310240 ]
        Key HDFS-4229 HADOOP-9152
        Affects Version/s 2.0.2-alpha [ 12322473 ]
        Affects Version/s 2.0.2-alpha [ 12322472 ]
        Component/s fs [ 12310689 ]
        Component/s namenode [ 12312926 ]
        Brock Noland made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Assignee Brock Noland [ brocknoland ]
        Brock Noland made changes -
        Field Original Value New Value
        Attachment HDFS-4229-0.patch.txt [ 12561135 ]
        Brock Noland created issue -

          People

          • Assignee:
            Brock Noland
            Reporter:
            Brock Noland
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development