[HADOOP-12173] NetworkTopology#add calls NetworkTopology#toString always

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.7.0
• Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
• Target Version/s: 2.7.4
• Hadoop Flags: Reviewed
• Assignee: Íñigo Goiri
• Reporter: Íñigo Goiri
• Created: 02/Jul/15 02:33
• Updated: 22/Feb/19 01:07
• Resolved: 03/Jul/15 07:52

Description

It always does a toString of the whole topology but this is not required when there are no errors. This is adding a very big overhead to large clusters as it's walking the whole tree every time we add a node to the cluster.

HADOOP-10953 did some fix in that area but the errors is still there.

Issue Links

• is related to: HADOOP-10953 NetworkTopology#add calls NetworkTopology#toString without holding the netlock
• relates to: HADOOP-14369 NetworkTopology calls expensive toString() when logging
• relates to: HADOOP-8159 NetworkTopology: getLeaf should check for invalid topologies
• relates to: HDFS-14186 blockreport storm slow down namenode restart seriously in large cluster
• relates to: HADOOP-12185 NetworkTopology is not efficient adding/getting/removing nodes