[HADOOP-13023] Distcp with -update feature on first time raw data not working

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.6.0
• Component/s: tools/distcp
• Assignee: Unassigned
• Reporter: Mavin Martin
• Created: 14/Apr/16 00:32
• Updated: 16/Apr/18 02:35

Description

When attempting to do a distcp with the -update feature toggled on encrypted data, the distcp shows as successful. Reading the encrypted file on the target_path does not work since the keyName does not exist.

Please see my example to reproduce the issue.

[root@xxx bin]# hdfs crypto -listZones
/tmp/a/ted                                DEF0000000000013
[root@xxx bin]# hdfs dfs -ls -R /tmp
drwxr-xr-x   - xxx xxx          0 2016-04-14 00:22 /tmp/a
drwxr-xr-x   - xxx xxx          0 2016-04-14 00:00 /tmp/a/ted
-rw-r--r--   3 xxx xxx         33 2016-04-14 00:00 /tmp/a/ted/test.txt
[root@xxx bin]# hadoop distcp -update /.reserved/raw/tmp/a/ted /.reserved/raw/tmp/a-with-update/ted
[root@xxx bin]# hdfs crypto -listZones
/tmp/a/ted                                DEF0000000000013
[root@xxx bin]# hadoop distcp /.reserved/raw/tmp/a/ted /.reserved/raw/tmp/a-no-update/ted
[root@xxx bin]# hdfs crypto -listZones
/tmp/a/ted                                DEF0000000000013
/tmp/a-no-update/ted                      DEF0000000000013

The crypto zone for 'a-with-update' should have been created since this is a new destination. You can verify this by looking at 'a-no-update'.