[HDFS-1094] Intelligent block placement policy to decrease probability of block loss - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Improvement
Status: Open
Priority: Major
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: None
Component/s: namenode
Labels:
None

Description

The current HDFS implementation specifies that the first replica is local and the other two replicas are on any two random nodes on a random remote rack. This means that if any three datanodes die together, then there is a non-trivial probability of losing at least one block in the cluster. This JIRA is to discuss if there is a better algorithm that can lower probability of losing a block.

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

prob.pdf
14/May/10 23:39
67 kB
Aravind Menon
prob.pdf
15/May/10 02:39
67 kB
Aravind Menon
failure_rate.py
19/Jul/10 01:55
11 kB
Rodrigo Schmidt
calculate_probs.py
19/Jul/10 01:57
2 kB
Rodrigo Schmidt

Issue Links

is blocked by

HDFS-1261 Important methods and fields of BlockPlacementPolicyDefault.java shouldn't be private

Open

is related to

HDFS-385 Design a pluggable interface to place replicas of blocks in HDFS

Closed

relates to

HBASE-5843 Improve HBase MTTR - Mean Time To Recover

Closed

Activity

People

Assignee:: Rodrigo Schmidt

Reporter:: Dhruba Borthakur

Votes:: 0 Vote for this issue

Watchers:: 44 Start watching this issue

Dates

Created:: 12/Apr/10 07:06

Updated:: 30/Jul/14 17:42