[MAPREDUCE-662] distcp -update fails if source directory is empty (i.e. no files to copy) and target directory does not exists.

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.21.0
• Fix Version/s: 0.21.0
• Component/s: distcp
• Assignee: Unassigned
• Reporter: Suhas
• Created: 25/Jun/09 17:58
• Updated: 24/Aug/10 21:14
• Resolved: 26/Jun/09 09:49

Description

It should either create empty target directory or not make any changes on the target (as there is nothing to copy from source), but it should return success.

Tested version of hadoop has HADOOP-5675.

-bash-3.1$ bin/hadoop distcp -Dmapred.job.queue.name=xxx -i -p -update -m 10 hftp://xxx.yyy.com:50070/user/gogate/mirror_test1 hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1

09/06/24 19:46:42 INFO tools.DistCp: srcPaths=[hftp://xxx.yyy.com:50070/user/gogate/mirror_test1]
09/06/24 19:46:42 INFO tools.DistCp: destPath=hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1
09/06/24 19:46:44 INFO tools.DistCp: hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1 does not exist.
09/06/24 19:46:44 INFO tools.DistCp: sourcePathsCount=1
09/06/24 19:46:44 INFO tools.DistCp: filesToCopyCount=0
09/06/24 19:46:44 INFO tools.DistCp: bytesToCopyCount=0.0
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.FileNotFoundException: File does not exist: hdfs://zzz.yyy.com:8020/user/gogate/mirror_test1
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:457)
at org.apache.hadoop.tools.DistCp.finalize(DistCp.java:706)
at org.apache.hadoop.tools.DistCp.copy(DistCp.java:653)
at org.apache.hadoop.tools.DistCp.run(DistCp.java:858)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.tools.DistCp.main(DistCp.java:885)