Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-8143

Change distcp to have -pb on by default

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha4
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      If -p option of distcp command is unspecified, block size is preserved.

      Description

      We should have the preserve blocksize (-pb) on in distcp by default.

      checksum which is on by default will always fail if blocksize is not the same.

        Attachments

        1. HADOOP-8143.1.patch
          2 kB
          Mithun Radhakrishnan
        2. HADOOP-8143.2.patch
          2 kB
          Mithun Radhakrishnan
        3. HADOOP-8143.3.patch
          2 kB
          Mithun Radhakrishnan

          Issue Links

          breaks

          Sub-task - The sub-task of the issue HADOOP-16756 distcp -update to S3A; abfs, etc always overwrites due to block size mismatch

          • Major - Major loss of function.
          • Resolved

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-16932 distcp copy calls getFileStatus() needlessly and can fail against S3

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved
          causes

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-17256 DistCp -update option will be invalid when distcp files from hdfs to S3

          • Major - Major loss of function.
          • Resolved
          is depended upon by

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-14557 Document HADOOP-8143 (Change distcp to have -pb on by default)

          • Trivial - Cosmetic problem like misspelt words or misaligned text.
          • Resolved
          is related to

          Bug - A problem which impairs or prevents the functions of the product. HIVE-16918 Skip ReplCopyTask distcp for _metadata copying. Also enable -pb for distcp

          • Major - Major loss of function.
          • Closed
          is superceded by

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-17044 Revert "HADOOP-8143. Change distcp to have -pb on by default"

          • Major - Major loss of function.
          • Resolved

            Activity

              People

              • Assignee:
                mithun Mithun Radhakrishnan
                Reporter:
                davet Dave Thompson
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: