[HDFS-11450] HDFS specific network topology classes with storage type info included

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.9.0, 3.0.0-alpha4
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Chen Liang
• Reporter: Chen Liang
• Created: 23/Feb/17 22:55
• Updated: 18/Aug/17 19:47
• Resolved: 02/Mar/17 17:25

Description

This JIRA adds storage type info into network topology.

More specifically, this JIRA adds a storage type map by extending InnerNodeImpl to describe the available storages under the current node's subtree. This map is updated when a node is added/removed from the subtree.

With this info, when choosing a random node with storage type requirement, the search could then decide to/not to go deeper into a subtree by examining the available storage types first.

One to-do item still, is that, we might still need to separately handle the cases where a Datanodes restarts, or a disk is hot-swapped, will file another JIRA in that case.

Issue Links

• is depended upon by: HDFS-11482 Add storage type demand to into DFSNetworkTopology#chooseRandom