[HDFS-10206] Datanodes not sorted properly by distance when the reader isn't a datanode

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha2
• Hadoop Flags: Reviewed
• Assignee: Nandakumar
• Reporter: Ming Ma
• Created: 24/Mar/16 17:04
• Updated: 10/Dec/16 01:57
• Resolved: 07/Dec/16 16:29

Description

If the DFSClient machine is not a datanode, but it shares its rack with some datanodes of the HDFS block requested, DatanodeManager#sortLocatedBlocks might not put the local-rack datanodes at the beginning of the sorted list. That is because the function didn't call networktopology.add(client); to properly set the node's parent node; something required by networktopology.sortByDistance to compute distance between two nodes in the same topology tree.

Another issue with networktopology.sortByDistance is it only distinguishes local rack from remote rack, but it doesn't support general distance calculation to tell how remote the rack is.

NetworkTopology.java
  protected int getWeight(Node reader, Node node) {
    // 0 is local, 1 is same rack, 2 is off rack
    // Start off by initializing to off rack
    int weight = 2;
    if (reader != null) {
      if (reader.equals(node)) {
        weight = 0;
      } else if (isOnSameRack(reader, node)) {
        weight = 1;
      }
    }
    return weight;
  }

HDFS-10203 has suggested moving the sorting from namenode to DFSClient to address another issue. Regardless of where we do the sorting, we still need fix the issues outline here.

Note that BlockPlacementPolicyDefault shares the same NetworkTopology object used by DatanodeManager and requires Nodes stored in the topology to be DatanodeDescriptor for block placement. So we need to make sure we don't pollute the NetworkTopology if we plan to fix it on the server side.