Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: distcp
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently distcp does a checksums check in addition to file length check to decide if a remote file has to be copied. If the number of files is high (thousands), this checksum check is proving to be fairly costly leading to a long time before the copy is started.

        Attachments

        1. mapred-1231.patch
          2 kB
          Jothi Padmanabhan
        2. mapred-1231-v1.patch
          5 kB
          Jothi Padmanabhan
        3. mapred-1231-v2.patch
          7 kB
          Jothi Padmanabhan
        4. mapred-1231-v3.patch
          8 kB
          Jothi Padmanabhan
        5. mapred-1231-v3.patch
          8 kB
          Jothi Padmanabhan
        6. mapred-1231-y20.patch
          4 kB
          Jothi Padmanabhan
        7. mapred-1231-y20-v2.patch
          7 kB
          Jothi Padmanabhan
        8. mapred-1231-y20-v3.patch
          8 kB
          Jothi Padmanabhan
        9. mapred-1231-y20-v4.patch
          8 kB
          Jothi Padmanabhan

          Activity

            People

            • Assignee:
              jothipn Jothi Padmanabhan
              Reporter:
              jothipn Jothi Padmanabhan
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: