[HDFS-2576] Namenode should have a favored nodes hint to enable clients to have control over block placement.

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.1.0-beta
• Component/s: hdfs-client, namenode
• Hadoop Flags: Reviewed
• Assignee: Devaraj Das
• Reporter: Pritam Damania
• Created: 21/Nov/11 18:27
• Updated: 27/Sep/15 13:37
• Resolved: 30/Apr/13 06:30

Description

Sometimes Clients like HBase are required to dynamically compute the datanodes it wishes to place the blocks for a file for higher level of locality. For this purpose there is a need of a way to give the Namenode a hint in terms of a favoredNodes parameter about the locations where the client wants to put each block. The proposed solution is a favored nodes parameter in the addBlock() method and in the create() file method to enable the clients to give the hints to the NameNode about the locations of each replica of the block. Note that this would be just a hint and finally the NameNode would look at disk usage, datanode load etc. and decide whether it can respect the hints or not.

Issue Links

• is related to: HDFS-4778 Invoke getPipeline in the chooseTarget implementation that has favoredNodes
• is related to: HDFS-6680 BlockPlacementPolicyDefault does not choose favored nodes correctly
• is related to: HDFS-4779 Backport HDFS-2576 to branch-1 (this is the favoredNodes API related changes)
• is required by: HBASE-5843 Improve HBase MTTR - Mean Time To Recover
• relates to: HDFS-4778 Invoke getPipeline in the chooseTarget implementation that has favoredNodes
• relates to: HBASE-4755 HBase based block placement in DFS