[HDFS-14216] NullPointerException happens in NamenodeWebHdfs

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 3.3.0, 3.2.1, 3.1.4, 2.10.2
• Assignee: lujie
• Reporter: lujie
• Created: 19/Jan/19 05:58
• Updated: 10/Sep/21 22:48
• Resolved: 03/Oct/19 02:51

Description

 workload

curl -i -X PUT -T $HOMEPARH/test.txt "http://hadoop1:9870/webhdfs/v1/input?op=CREATE&excludedatanodes=hadoop2"

the method

org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(String excludeDatanodes){

    HashSet<Node> excludes = new HashSet<Node>();
    if (excludeDatanodes != null) {
       for (String host : StringUtils
         .getTrimmedStringCollection(excludeDatanodes)) {
         int idx = host.indexOf(":");
       if (idx != -1) { 
            excludes.add(bm.getDatanodeManager().getDatanodeByXferAddr(
           host.substring(0, idx), Integer.parseInt(host.substring(idx + 1))));
       } else {
          excludes.add(bm.getDatanodeManager().getDatanodeByHost(host));//line280
       }
      }
    }

}

when datanode(e.g.hadoop2) is just  wiped before line280, or we give the wrong DN name*,then  bm.getDatanodeManager().getDatanodeByHost(host) will return null, *excludes containes null. while excludes are used later, NPE happens:

java.lang.NullPointerException
at org.apache.hadoop.net.NodeBase.getPath(NodeBase.java:113)
at org.apache.hadoop.net.NetworkTopology.countNumOfAvailableNodes(NetworkTopology.java:672)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:533)
at org.apache.hadoop.net.NetworkTopology.chooseRandom(NetworkTopology.java:491)
at org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.chooseDatanode(NamenodeWebHdfsMethods.java:323)
at org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.redirectURI(NamenodeWebHdfsMethods.java:384)
at org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods.put(NamenodeWebHdfsMethods.java:652)
at org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:600)
at org.apache.hadoop.hdfs.server.namenode.web.resources.NamenodeWebHdfsMethods$2.run(NamenodeWebHdfsMethods.java:597)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:73)
at org.apache.hadoop.ipc.ExternalCall.run(ExternalCall.java:30)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1876)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2830)

 

Issue Links

• links to: GitHub Pull Request #498