VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 0.20.1
    • 0.21.0
    • distcp
    • None
    • Reviewed

    Description

      Currently distcp does a checksums check in addition to file length check to decide if a remote file has to be copied. If the number of files is high (thousands), this checksum check is proving to be fairly costly leading to a long time before the copy is started.

      Attachments

        1. mapred-1231-y20-v4.patch
          8 kB
          Jothi Padmanabhan
        2. mapred-1231-y20-v3.patch
          8 kB
          Jothi Padmanabhan
        3. mapred-1231-y20-v2.patch
          7 kB
          Jothi Padmanabhan
        4. mapred-1231-y20.patch
          4 kB
          Jothi Padmanabhan
        5. mapred-1231-v3.patch
          8 kB
          Jothi Padmanabhan
        6. mapred-1231-v3.patch
          8 kB
          Jothi Padmanabhan
        7. mapred-1231-v2.patch
          7 kB
          Jothi Padmanabhan
        8. mapred-1231-v1.patch
          5 kB
          Jothi Padmanabhan
        9. mapred-1231.patch
          2 kB
          Jothi Padmanabhan

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jothipn Jothi Padmanabhan
            jothipn Jothi Padmanabhan
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment