[HDFS-1094] Intelligent block placement policy to decrease probability of block loss

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: namenode
• Assignee: Rodrigo Schmidt
• Reporter: Dhruba Borthakur
• Created: 12/Apr/10 07:06
• Updated: 30/Jul/14 17:42

Description

The current HDFS implementation specifies that the first replica is local and the other two replicas are on any two random nodes on a random remote rack. This means that if any three datanodes die together, then there is a non-trivial probability of losing at least one block in the cluster. This JIRA is to discuss if there is a better algorithm that can lower probability of losing a block.

Issue Links

• is blocked by: HDFS-1261 Important methods and fields of BlockPlacementPolicyDefault.java shouldn't be private
• is related to: HDFS-385 Design a pluggable interface to place replicas of blocks in HDFS
• relates to: HBASE-5843 Improve HBase MTTR - Mean Time To Recover