Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3873

DistCp should have an option for limiting the number of files/bytes being copied

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.19.0
    • None
    • None
    • Reviewed
    • Added two new options -filelimit <n> and -sizelimit <n> to DistCp for limiting the total number of files and the total size in bytes, respectively.

    Description

      A single DistCp command may potentially copies a huge number of files/bytes. In such case, DistCp will run a long time and there is no way stop it nicely. It would be good if DistCp have an option to limit the number of files/bytes being copied. Once the limit is reached, DistCp will terminate and return success. All files copied are guaranteed to be good and there is no partially copied file.

      Attachments

        1. 3873_20080811b.patch
          31 kB
          Tsz-wo Sze
        2. 3873_20080811b_0.18.patch
          31 kB
          Tsz-wo Sze
        3. 3873_20080808b.patch
          14 kB
          Tsz-wo Sze

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            szetszwo Tsz-wo Sze
            szetszwo Tsz-wo Sze
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment