[HADOOP-3084] distcp fails for files with zero length

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.16.0, 0.16.1, 0.16.2
• Fix Version/s: 0.16.2
• Component/s: util
• Assignee: Christopher Douglas
• Reporter: Mukund Madhugiri
• Created: 24/Mar/08 23:17
• Updated: 03/Apr/08 15:42
• Resolved: 27/Mar/08 22:37

Description

distcp fails for files with zero length. This is a regression from 0.15.3

distcp hftp://<namenode:port>/dir1/file1 file2
08/03/24 23:09:45 INFO util.CopyFiles: srcPaths=[hftp://<namenode:port>/dir1/file1]
08/03/24 23:09:45 INFO util.CopyFiles: destPath=file2
08/03/24 23:09:45 INFO util.CopyFiles: srcCount=1
08/03/24 23:09:46 INFO mapred.JobClient: Running job: job_200803242306_0001
08/03/24 23:09:47 INFO mapred.JobClient: map 0% reduce 0%
08/03/24 23:10:01 INFO mapred.JobClient: Task Id : task_200803242306_0001_m_000000_0, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.close(CopyFiles.java:448)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:53)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)

08/03/24 23:10:18 INFO mapred.JobClient: Task Id : task_200803242306_0001_m_000000_1, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.close(CopyFiles.java:448)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:53)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)

08/03/24 23:10:33 INFO mapred.JobClient: Task Id : task_200803242306_0001_m_000000_2, Status : FAILED
java.io.IOException: Copied: 0 Skipped: 0 Failed: 1
at org.apache.hadoop.util.CopyFiles$FSCopyFilesMapper.close(CopyFiles.java:448)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:53)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:208)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2084)

^@08/03/24 23:10:48 INFO mapred.JobClient: map 100% reduce 100%
With failures, global counters are inaccurate; consider running with -i
Copy failed: java.io.IOException: Job failed!
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:894)
at org.apache.hadoop.util.CopyFiles.copy(CopyFiles.java:526)
at org.apache.hadoop.util.CopyFiles.run(CopyFiles.java:596)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
at org.apache.hadoop.util.CopyFiles.main(CopyFiles.java:612)