[HDFS-4521] invalid network topologies should not be cached

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.1.0-beta, 1.3.0
• Fix Version/s: 2.1.0-beta, 1.3.0
• Target Version/s: 2.1.0-beta, 1.3.0
• Assignee: Colin McCabe
• Reporter: Colin McCabe
• Created: 22/Feb/13 01:41
• Updated: 25/Oct/19 20:25
• Resolved: 15/Jul/13 22:20

Description

When the network topology is invalid, the DataNode refuses to start with a message such as this:

org.apache.hadoop.hdfs.server.protocol.DatanodeProtocol.registerDatanode from 172.29.122.23:55886: error:
org.apache.hadoop.net.NetworkTopology$InvalidTopologyException: Invalid network topology. You cannot have a rack and a non-rack node at the same level of the network topology.

This is expected if you specify a topology file or script which puts leaf nodes at two different depths. However, one problem we have now is that this incorrect topology is cached forever. Once the NameNode sees it, this DataNode can never be added to the cluster, since this exception will be rethrown each time. The NameNode will not check to see if the topology file or script has changed. We should clear the topology mappings when there is an InvalidTopologyException, to prevent this problem.

Issue Links

• breaks: HDFS-10865 Datanodemanager adds nodes twice to NetworkTopology
• is duplicated by: HADOOP-9633 An incorrect data node might be added to the network topology, an exception is thrown though
• is related to: HADOOP-9998 Provide methods to clear only part of the DNSToSwitchMapping
• relates to: HADOOP-9633 An incorrect data node might be added to the network topology, an exception is thrown though
• relates to: HDFS-14247 Repeat adding node description into network topology