Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.1
    • Fix Version/s: 0.21.0
    • Component/s: distcp
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently distcp does a checksums check in addition to file length check to decide if a remote file has to be copied. If the number of files is high (thousands), this checksum check is proving to be fairly costly leading to a long time before the copy is started.

      1. mapred-1231.patch
        2 kB
        Jothi Padmanabhan
      2. mapred-1231-y20.patch
        4 kB
        Jothi Padmanabhan
      3. mapred-1231-v1.patch
        5 kB
        Jothi Padmanabhan
      4. mapred-1231-y20-v2.patch
        7 kB
        Jothi Padmanabhan
      5. mapred-1231-v2.patch
        7 kB
        Jothi Padmanabhan
      6. mapred-1231-v3.patch
        8 kB
        Jothi Padmanabhan
      7. mapred-1231-y20-v3.patch
        8 kB
        Jothi Padmanabhan
      8. mapred-1231-v3.patch
        8 kB
        Jothi Padmanabhan
      9. mapred-1231-y20-v4.patch
        8 kB
        Jothi Padmanabhan

        Activity

          People

          • Assignee:
            Jothi Padmanabhan
            Reporter:
            Jothi Padmanabhan
          • Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development