[HADOOP-8143] Change distcp to have -pb on by default

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 3.0.0-alpha4
• Hadoop Flags: Incompatible change, Reviewed
• Release Note: If -p option of distcp command is unspecified, block size is preserved.
• Assignee: Mithun Radhakrishnan
• Reporter: Dave Thompson
• Created: 06/Mar/12 16:17
• Updated: 10/Sep/20 11:09
• Resolved: 21/May/20 12:54

Description

We should have the preserve blocksize (-pb) on in distcp by default.

checksum which is on by default will always fail if blocksize is not the same.

Issue Links

• breaks: HADOOP-16756 distcp -update to S3A; abfs, etc always overwrites due to block size mismatch
• breaks: HADOOP-16932 distcp copy calls getFileStatus() needlessly and can fail against S3
• causes: HADOOP-17256 DistCp -update option will be invalid when distcp files from hdfs to S3
• is depended upon by: HADOOP-14557 Document HADOOP-8143 (Change distcp to have -pb on by default)
• is related to: HIVE-16918 Skip ReplCopyTask distcp for _metadata copying. Also enable -pb for distcp
• is superceded by: HADOOP-17044 Revert "HADOOP-8143. Change distcp to have -pb on by default"