[MAPREDUCE-5014] Extending DistCp through a custom CopyListing is not possible

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0, 0.23.1, 0.23.3, 0.23.4, 0.23.5
• Fix Version/s: 2.5.0
• Component/s: distcp
• Target Version/s: 0.23.0, 0.23.1, 0.23.3, 0.23.4, 0.23.5
• Hadoop Flags: Reviewed
• Assignee: Srikanth Sundarrajan
• Reporter: Srikanth Sundarrajan
• Created: 20/Feb/13 01:34
• Updated: 10/Mar/15 04:30
• Resolved: 22/Mar/13 10:41

Description

• While it is possible to implement a custom CopyListing in DistCp, DistCp driver class doesn't allow for using this custom CopyListing.

• Allow SimpleCopyListing to provide an option to exclude files (For instance it is useful to exclude FileOutputCommiter.SUCCEEDED_FILE_NAME during copy as premature copy can indicate that the entire data is available at the destination)