Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10206

Datanodes not sorted properly by distance when the reader isn't a datanode

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.9.0, 3.0.0-alpha2
    • None
    • None
    • Reviewed

    Description

      If the DFSClient machine is not a datanode, but it shares its rack with some datanodes of the HDFS block requested, DatanodeManager#sortLocatedBlocks might not put the local-rack datanodes at the beginning of the sorted list. That is because the function didn't call networktopology.add(client); to properly set the node's parent node; something required by networktopology.sortByDistance to compute distance between two nodes in the same topology tree.

      Another issue with networktopology.sortByDistance is it only distinguishes local rack from remote rack, but it doesn't support general distance calculation to tell how remote the rack is.

      NetworkTopology.java
        protected int getWeight(Node reader, Node node) {
          // 0 is local, 1 is same rack, 2 is off rack
          // Start off by initializing to off rack
          int weight = 2;
          if (reader != null) {
            if (reader.equals(node)) {
              weight = 0;
            } else if (isOnSameRack(reader, node)) {
              weight = 1;
            }
          }
          return weight;
        }
      

      HDFS-10203 has suggested moving the sorting from namenode to DFSClient to address another issue. Regardless of where we do the sorting, we still need fix the issues outline here.

      Note that BlockPlacementPolicyDefault shares the same NetworkTopology object used by DatanodeManager and requires Nodes stored in the topology to be DatanodeDescriptor for block placement. So we need to make sure we don't pollute the NetworkTopology if we plan to fix it on the server side.

      Attachments

        1. HDFS-10206-branch-2.8.003.patch
          11 kB
          Nandakumar
        2. HDFS-10206.003.patch
          11 kB
          Nandakumar
        3. HDFS-10206.002.patch
          11 kB
          Nandakumar
        4. HDFS-10206.001.patch
          11 kB
          Nandakumar
        5. HDFS-10206.000.patch
          6 kB
          Nandakumar

        Activity

          People

            nanda Nandakumar
            mingma Ming Ma
            Votes:
            0 Vote for this issue
            Watchers:
            10 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: