Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-3325

When configuring "dfs.namenode.safemode.threshold-pct" to a value greater or equal to 1 there is mismatch in the UI report

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.0.0-alpha
    • 2.8.0, 3.0.0-alpha1
    • None
    • None
    • Reviewed

    Description

      When dfs.namenode.safemode.threshold-pct is configured to n
      Namenode will be in safemode until n percentage of blocks that should satisfy
      the minimal replication requirement defined by dfs.namenode.replication.min is reported to namenode

      But in UI it displays that n percentage of total blocks + 1 blocks are additionally needed
      to come out of the safemode

      Scenario 1:
      ============
      Configurations:
      dfs.namenode.safemode.threshold-pct = 2
      dfs.replication = 2
      dfs.namenode.replication.min =2
      Step 1: Start NN,DN1,DN2
      Step 2: Write a file "a.txt" which has got 167 blocks
      step 3: Stop NN,DN1,DN2
      Step 4: start NN
      In UI report the Number of blocks needed to come out of safemode and number of blocks actually present is different.

      Cluster Summary
      Security is OFF 
      Safe mode is ON. The reported blocks 0 needs additional 335 blocks to reach the threshold 2.0000 of total blocks 167. Safe mode will be turned off automatically.
      2 files and directories, 167 blocks = 169 total.
      Heap Memory used 57.05 MB is 2% of Commited Heap Memory 2 GB. Max Heap Memory is 2 GB. 
      Non Heap Memory used 23.37 MB is 17% of Commited Non Heap Memory 130.44 MB. Max Non Heap Memory is 176 MB.

      Scenario 2:
      ===========
      Configurations:
      dfs.namenode.safemode.threshold-pct = 1
      dfs.replication = 2
      dfs.namenode.replication.min =2
      Step 1: Start NN,DN1,DN2
      Step 2: Write a file "a.txt" which has got 167 blocks
      step 3: Stop NN,DN1,DN2
      Step 4: start NN
      In UI report the Number of blocks needed to come out of safemode and number of blocks actually present is different

      Cluster Summary
      Security is OFF 
      Safe mode is ON. The reported blocks 0 needs additional 168 blocks to reach the threshold 1.0000 of total blocks 167. Safe mode will be turned off automatically.
      2 files and directories, 167 blocks = 169 total.
      Heap Memory used 56.2 MB is 2% of Commited Heap Memory 2 GB. Max Heap Memory is 2 GB. 
      Non Heap Memory used 23.37 MB is 17% of Commited Non Heap Memory 130.44 MB. Max Non Heap Memory is 176 MB.

      Attachments

        1. HDFS-3325.1.patch
          1 kB
          J.Andreina
        2. HDFS-3325.2.patch
          3 kB
          J.Andreina

        Issue Links

          Activity

            People

              andreina J.Andreina
              andreina J.Andreina
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: