Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9613

Avoid checking file checksums after copy when possible

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Patch Available
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • None
    • None

    Description

      While working on related issue, it was noticed there are some places in distcp that's better to be improved and cleaned up. Particularly, after a file is coped to target cluster, it will check the copied file is fine or not. For replicated files, when checking, if the source block size and checksum option are not preserved while copying, we can avoid comparing the file checksums, which may save some time.

      Attachments

        1. HDFS-9613-v1.patch
          14 kB
          Kai Zheng
        2. HDFS-9613-v2.patch
          18 kB
          Kai Zheng

        Activity

          People

            drankye Kai Zheng
            drankye Kai Zheng
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated: