[HADOOP-8820] Backport HADOOP-8469 and HADOOP-8470: add "NodeGroup" layer in new NetworkTopology (also known as NetworkTopologyWithNodeGroup)

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0
• Fix Version/s: 1.2.0, 1-win
• Component/s: net
• Target Version/s: 1.2.0
• Hadoop Flags: Reviewed
• Assignee: Junping Du
• Reporter: Junping Du
• Created: 16/Sep/12 18:43
• Updated: 15/May/13 05:16
• Resolved: 13/Nov/12 21:39

Description

This patch backport HADOOP-8469 and HADOOP-8470 to branch-1 and includes:
1. Make NetworkTopology class pluggable for extension.
2. Implement a 4-layer NetworkTopology class (named as NetworkTopologyWithNodeGroup) to use in virtualized environment (or other situation with additional layer between host and rack).

Issue Links

• blocks: HDFS-3941 Backport HDFS-3498 and HDFS3601: update replica placement policy for new added "NodeGroup" layer topology
• blocks: HDFS-3942 Backport HDFS-3495: Update balancer policy for Network Topology with additional 'NodeGroup' layer
• blocks: MAPREDUCE-4660 Update task placement policy for NetworkTopology with 'NodeGroup' layer
• Is contained by: HADOOP-8817 Backport Network Topology Extension for Virtualization (HADOOP-8468) to branch-1