[MAPREDUCE-6414] Distcp command very slow to enumerate files needing

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.5.0
• Component/s: distcp

Environment

RHEL 6.5

• Assignee: Unassigned
• Reporter: Tyler Hale
• Created: 23/Jun/15 20:29
• Updated: 23/Jun/15 20:29

Description

When copying large amounts of data using distcp utility (100's of TBs), the distcp utility takes a large time to enumerate all of the files that have changed. In my system, this corresponds to 14-16 hours before the actual copying of data begins.