[HDFS-3498] Make Replica Removal Policy pluggable and ReplicaPlacementPolicyDefault extensible for reusing code in subclass

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.0, 2.0.0-alpha
• Fix Version/s: 1.2.0, 2.1.0-beta
• Component/s: namenode
• Target Version/s: 2.0.5-alpha
• Hadoop Flags: Reviewed
• Assignee: Junping Du
• Reporter: Junping Du
• Created: 04/Jun/12 05:34
• Updated: 27/Aug/13 22:07
• Resolved: 03/Jun/13 04:35

Description

ReplicaPlacementPolicy is already a pluggable component in Hadoop. However, the Replica Removal Policy is still nested in BlockManager that need to be separated out into a ReplicaPlacementPolicy then can be override later. Also it looks like hadoop unit test lack the testing on replica removal policy, so we add it here.
On the other hand, as a implementation of ReplicaPlacementPolicy, ReplicaPlacementDefault still show lots of generic for other topology cases like: virtualization, and we want to make code in ReplicaPlacementPolicyDefault can be reused as much as possible so a few of its methods were changed from private to protected.

Issue Links

• is depended upon by: HDFS-4240 In nodegroup-aware case, make sure nodes are avoided to place replica if some replica are already under the same nodegroup
• is depended upon by: HDFS-3601 Implementation of ReplicaPlacementPolicyNodeGroup to support 4-layer network topology
• is part of: HADOOP-8468 Umbrella of enhancements to support different failure and locality topologies
• is related to: HDFS-3941 Backport HDFS-3498 and HDFS3601: update replica placement policy for new added "NodeGroup" layer topology