Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
-
Added two new options -filelimit <n> and -sizelimit <n> to DistCp for limiting the total number of files and the total size in bytes, respectively.
Description
A single DistCp command may potentially copies a huge number of files/bytes. In such case, DistCp will run a long time and there is no way stop it nicely. It would be good if DistCp have an option to limit the number of files/bytes being copied. Once the limit is reached, DistCp will terminate and return success. All files copied are guaranteed to be good and there is no partially copied file.
Attachments
Attachments
Issue Links
- is depended upon by
-
HADOOP-3942 Update DistCp documentation
- Closed
- relates to
-
HADOOP-3939 DistCp should support an option for deleting non-existing files.
- Closed