Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-4521

invalid network topologies should not be cached

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.1.0-beta, 1.3.0
    • Fix Version/s: 2.1.0-beta, 1.3.0
    • Component/s: None
    • Labels:
      None

      Description

      When the network topology is invalid, the DataNode refuses to start with a message such as this:

      org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.registerDatanode from 172.29.122.23:55886: error:
      org.apache.hadoop.net.NetworkTopology$InvalidTopologyException: Invalid network topology. You cannot have a rack and a non-rack node at the same level of the network topology.

      This is expected if you specify a topology file or script which puts leaf nodes at two different depths. However, one problem we have now is that this incorrect topology is cached forever. Once the NameNode sees it, this DataNode can never be added to the cluster, since this exception will be rethrown each time. The NameNode will not check to see if the topology file or script has changed. We should clear the topology mappings when there is an InvalidTopologyException, to prevent this problem.

        Attachments

        1. HDFS-4521-branch1.patch
          21 kB
          Junping Du
        2. HDFS-4521.008.patch
          29 kB
          Colin McCabe
        3. HDFS-4521.006.patch
          15 kB
          Colin McCabe
        4. HDFS-4521.005.patch
          13 kB
          Colin McCabe
        5. HDFS-4521.002.patch
          94 kB
          Colin McCabe
        6. HDFS-4521.001.patch
          83 kB
          Colin McCabe

          Issue Links

            Activity

              People

              • Assignee:
                cmccabe Colin McCabe
                Reporter:
                cmccabe Colin McCabe
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: