Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-2767

org.apache.hadoop.net.NetworkTopology.InnerNode#getLeaf does not return the last node on a rack when used with an excluded node

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • 0.15.3
    • 0.17.0
    • None
    • None

    Description

      I have written some test code that shows NetworkTopology.InnerNode#getLeaf will never return the last node on the rack if it is called with an excludedNode (for example the first node on the rack).

      Consequently I suspect that NetworkTopology.chooseRandom() will never returns the last node on the remote rack for the second replica in DFS.

      I have some test code that demonstrates this problem at the getLeaf level, although it is necessary to change the visibility of the NetworkTopology.InnerNode, NetworkTopology.InnerNode#getLeaf and NetworkTopology.getNode from private to package default to run the test.

      TODO: Demonstrate problem at NetworkTopology.chooseRandom level, then submit candidate fix.

      Attachments

        1. NetworkTopologyTest.java
          3 kB
          Mark Butler
        2. excludedLeaf.patch
          3 kB
          Hairong Kuang
        3. excludedLeaf2.patch
          6 kB
          Mark Butler
        4. excludedLeaf3.patch
          6 kB
          Hairong Kuang
        5. excludedLeaf4.patch
          6 kB
          Hairong Kuang

        Activity

          People

            hairong Hairong Kuang
            butlermh Mark Butler
            Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: