[HDFS-3564] Design enhancements to the pluggable blockplacementpolicy

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1-win
• Component/s: namenode
• Hadoop Flags: Reviewed
• Assignee: Sumadhur Reddy Bolli
• Reporter: Sumadhur Reddy Bolli
• Created: 25/Jun/12 23:49
• Updated: 11/Sep/13 00:25
• Resolved: 17/Sep/12 14:23

Description

Currently, it is assumed that no placement policy other than the default can work with the balancer and that the minimum racks required for any placement policy is always 2. The attached pdf suggests enhancements to the existing block placement policy abstraction to enable the following
a) Let placement policies decide if they are compatible with balancer
b) Provide an api to move blocks for balancing
c) Let the placement policy decide the minimum number of racks given the replication factor
d) Also, the private methods in the default policy are made protected, similar to the way its done in trunk, to enable easy derivation of custom policies.
Please refer the pdf and the patch for details.

Issue Links

• blocks: HDFS-3566 Custom Replication Policy for Azure
• is blocked by: HDFS-3649 Port HDFS-385 to branch-1-win
• is related to: HADOOP-8468 Umbrella of enhancements to support different failure and locality topologies
• is related to: HADOOP-8079 Proposal for enhancements to Hadoop for Windows Server and Windows Azure development and runtime environments
• is related to: HDFS-385 Design a pluggable interface to place replicas of blocks in HDFS