[HDFS-7497] Inconsistent report of decommissioning DataNodes between dfsadmin and NameNode webui

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.7.0
• Component/s: datanode, namenode
• Labels: supportability
• Assignee: Yongjun Zhang
• Reporter: Yongjun Zhang
• Created: 09/Dec/14 05:20
• Updated: 24/Apr/15 23:22
• Resolved: 12/Dec/14 02:14

Description

It's observed that dfsadmin report list DNs in the decomm state while NN UI list DNs in dead state.

I found what happens is:

NN webui uses two steps to get the result:

• first collect a list of all alive DNs,
• traverse through all live DNs to find decommissioning DNs.

It calls the following method to decide whether a DN is dead or alive:

  /** Is the datanode dead? */
  boolean isDatanodeDead(DatanodeDescriptor node) {
    return (node.getLastUpdate() <
            (Time.now() - heartbeatExpireInterval));
  }

On the other hand, dfsadmin traverse all DNs to find to all decommissioning DNs (check whether a DN is in AdminStates.DECOMMISSION_INPROGRESS state), without checking whether a DN is dead or alive like above.

The problem is, when a DN is determined to be dead, its state may still be AdminStates.DECOMMISSION_INPROGRESS .