Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6397

NN shows inconsistent value in deadnode count

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.4.1
    • Fix Version/s: 2.4.1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Context:
      When NN is started , without any live datanode but there are nodes in the dfs.includes, NN shows the deadcount as '0'.

      There are two inconsistencies:
      1. If you click on deadnode links (which shows the count is 0), it will display the list of deadnodes correctly.
      2. hadoop 1.x used to display the count correctly.

      The following snippets of JMX response will explain it further:
      Look at the value of "NumDeadDataNodes"

       {
          "name" : "Hadoop:service=NameNode,name=FSNamesystemState",
          "modelerType" : "org.apache.hadoop.hdfs.server.namenode.FSNamesystem",
          "CapacityTotal" : 0,
          "CapacityUsed" : 0,
      ... 
         "NumLiveDataNodes" : 0,
          "NumDeadDataNodes" : 0,
          "NumDecomLiveDataNodes" : 0,
          "NumDecomDeadDataNodes" : 0,
          "NumDecommissioningDataNodes" : 0,
          "NumStaleDataNodes" : 0
        },
      

      Look at " "DeadNodes"".

      {
          "name" : "Hadoop:service=NameNode,name=NameNodeInfo",
          "modelerType" : "org.apache.hadoop.hdfs.server.namenode.FSNamesystem",
          
      ....
          "TotalBlocks" : 70,
          "TotalFiles" : 129,
          "NumberOfMissingBlocks" : 0,
          "LiveNodes" : "{}",
          "DeadNodes" : "{\"<MMMMM>.linkedin.com\":{\"lastContact\":1400037397,\"decommissioned\":false,\"xferaddr\":\"172.XX.X.XX:71\"},\"<NNNNN>.linkedin.com\":{\"lastContact\":1400037397,\"decommissioned\":false,\"xferaddr\":\"172.XX.XX.XX:71\"}}",
          "DecomNodes" : "{}",
         .....
        }
      

        Attachments

        1. HDFS-6397.1.patch
          2 kB
          Mohammad Islam
        2. HDFS-6397.2.patch
          5 kB
          Mohammad Islam
        3. HDFS-6397.3.patch
          3 kB
          Mohammad Islam
        4. HDFS-6397.branch-2.merge.patch
          4 kB
          Kihwal Lee

          Activity

            People

            • Assignee:
              kamrul Mohammad Islam
              Reporter:
              kamrul Mohammad Islam
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: