[MAPREDUCE-2117] Superfast Distcp when copying data within the same hdfs cluster

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: distcp
• Assignee: Unassigned
• Reporter: Dhruba Borthakur
• Created: 08/Oct/10 05:39
• Updated: 08/Oct/10 20:13

Description

There are use cases when distcp is used to copy a bunch of files/directories from one part of the HDFS namespace to another part within the same HDFS cluster. It is superfast if we can instruct relevant datanodes to make local replicas of relevant blocks and limit network usage to a minimum. It is especially useful to make HBase take a backup of a region with minimum downtime.

Issue Links

• is related to: HDFS-222 Support for concatenating of files into a single file