[HDFS-4240] In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.2.0, 2.1.0-beta
• Component/s: namenode
• Target Version/s: 2.0.5-alpha
• Hadoop Flags: Reviewed
• Assignee: Junping Du
• Reporter: Junping Du
• Created: 15/Nov/12 16:14
• Updated: 27/Aug/13 22:08
• Resolved: 03/Jun/13 05:51

Description

In previous implementation for HADOOP-8468, 3rd replica is avoid to place on the same nodegroup of 2nd replica. But it didn't provide check on nodegroup of 1st replica, so if 2nd replica's rack is not efficient to place replica, then it is possible to place 3rd and 1st replica within the same node group. We need a change to remove all nodes from available nodes for placing replica if there already replica on the same nodegroup.

Issue Links

• depends upon: HDFS-3601 Implementation of ReplicaPlacementPolicyNodeGroup to support 4-layer network topology
• depends upon: HDFS-3498 Make Replica Removal Policy pluggable and ReplicaPlacementPolicyDefault extensible for reusing code in subclass
• is part of: HADOOP-8468 Umbrella of enhancements to support different failure and locality topologies
• relates to: HDFS-4337 Backport HDFS-4240 to branch-1: Make sure nodes are avoided to place replica if some replica are already under the same nodegroup.