[HDFS-6454] Configurable Block Placement Policy

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: namenode
• Assignee: Unassigned
• Reporter: Guo Ruijing
• Created: 28/May/14 09:48
• Updated: 28/May/14 09:48

Description

In existing implementation, block choose priority is localhost/remote rack/local rack/ramdon.
in BlockPlacementPolicyDefault, network topology is /rack/host.
In BlockPlacementPolicyWithNodeGroup, network topology is /rack/nodegroup/host.

This JIRA is to propose block choose priority can be configurable as:

<property>
<name>dfs.block.replicator.priority</name>
<value>0, 2, 1, *</value>
<description>
default network topology is /level2/level1
nodegroup network topology is /level3/level2/level1. choose priority can be 0(localhost), 3(remote rack), 2(local rack), *(any host)
</description>
</property>

Another example (one VM includes serveral dockers/containers) so network topology can be /rack/nodegroup/container/host. in this case, block replicator priority can be
0(localhost), 4(remote rack), 3(local rack), *(any host)