Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14491

More Clarity on Namenode UI Around Blocks and Replicas

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 3.3.0, 3.2.1, 3.1.3
    • None
    • None

    Description

      I recently deleted more than 1/3 of the files in my HDFS installation. During the process of the delete, I noticed that the NameNode UI near the top has a line like this:

      44,031,342 files and directories, 38,988,775 blocks = 83,020,117 total filesystem object(s).

      Then lower down had a line like this:

      Number of Blocks Pending Deletion 40000000

      That made it appear that I was deleting more blocks than exist in the cluster. When that number was below the total number of blocks, I briefly believed I had deleted the entire cluster. In reality, the second number includes replicas, while the first does not.

      The UI should be clarified to indicate where "Blocks" includes replicas and where it doesn't. This may also have an impact on the under-replicated count.

      Attachments

        1. HDFS-14491.001.patch
          1.0 kB
          Siyao Meng

        Activity

          People

            smeng Siyao Meng
            jackowaya Alan Jackoway
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: