Details
-
Sub-task
-
Status: Patch Available
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
This JIRA is to add a per-file "over-replication" factor to HDFS. As mentioned in HDFS-13069, the over-replication factor will be the excess replicas that will be allowed to exist for a file or block. This is beneficial if the application deems additional replicas for a file are needed. In the case of HDFS-13069, it would allow copies of data in PROVIDED storage to be cached locally in HDFS in a read-through manner.
The Namenode will not proactively meet the over-replication i.e., it does not schedule replications if the number of replicas for a block is less than (replication factor + over-replication factor) as long as they are more than the replication factor of the file.