[HDFS-13088] Allow HDFS files/blocks to be over-replicated.

• Type: Sub-task
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Assignee: Virajith Jalaparti
• Reporter: Virajith Jalaparti
• Created: 30/Jan/18 19:51
• Updated: 07/Aug/18 23:38

Description

This JIRA is to add a per-file "over-replication" factor to HDFS. As mentioned in HDFS-13069, the over-replication factor will be the excess replicas that will be allowed to exist for a file or block. This is beneficial if the application deems additional replicas for a file are needed. In the case of HDFS-13069, it would allow copies of data in PROVIDED storage to be cached locally in HDFS in a read-through manner.

The Namenode will not proactively meet the over-replication i.e., it does not schedule replications if the number of replicas for a block is less than (replication factor + over-replication factor) as long as they are more than the replication factor of the file.