[HDFS-9843] Document distcp options required for copying between encrypted locations

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 3.0.0-alpha1
• Component/s: distcp, documentation, encryption
• Hadoop Flags: Reviewed
• Assignee: Xiaoyu Yao
• Reporter: Xiaoyu Yao
• Created: 22/Feb/16 22:53
• Updated: 30/Aug/16 01:19
• Resolved: 24/Feb/16 23:28

Description

In TransparentEncryption.md#Distcp_considerations document section, we have "Copying_between_encrypted_and_unencrypted_locations" which requires -skipcrccheck and -update.

These options should be documented as required for "Copying between encrypted locations" use cases as well because this involves decrypting source file and encrypting destination file with a different EDEK, resulting in different checksum at the destination. Distcp will fail at crc check if -skipcrccheck if not specified.

This ticket is opened to document the required options for "Copying between encrypted locations" use cases when using distcp with HDFS encryption.