Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14491

More Clarity on Namenode UI Around Blocks and Replicas

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.3.0, 3.2.1, 3.1.3
    • Component/s: None
    • Labels:
      None

      Description

      I recently deleted more than 1/3 of the files in my HDFS installation. During the process of the delete, I noticed that the NameNode UI near the top has a line like this:

      44,031,342 files and directories, 38,988,775 blocks = 83,020,117 total filesystem object(s).

      Then lower down had a line like this:

      Number of Blocks Pending Deletion 40000000

      That made it appear that I was deleting more blocks than exist in the cluster. When that number was below the total number of blocks, I briefly believed I had deleted the entire cluster. In reality, the second number includes replicas, while the first does not.

      The UI should be clarified to indicate where "Blocks" includes replicas and where it doesn't. This may also have an impact on the under-replicated count.

        Attachments

        1. HDFS-14491.001.patch
          1.0 kB
          Siyao Meng

          Activity

            People

            • Assignee:
              smeng Siyao Meng
              Reporter:
              jackowaya Alan Jackoway
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: