[HDFS-1085] hftp read failing silently

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.22.0
• Component/s: datanode
• Hadoop Flags: Reviewed
• Assignee: Tsz-wo Sze
• Reporter: Koji Noguchi
• Created: 06/Apr/10 22:49
• Updated: 16/Jul/12 17:57
• Resolved: 19/Jul/10 21:00

Description

When performing a massive distcp through hftp, we saw many tasks fail with

2010-04-06 17:56:43,005 INFO org.apache.hadoop.tools.DistCp: FAIL 2010/0/part-00032 : java.io.IOException: File size not matched: copied 193855488 bytes (184.9m) to tmpfile (=hdfs://omehost.com:8020/somepath/part-00032)
but expected 1710327403 bytes (1.6g) from hftp://someotherhost/somepath/part-00032
at org.apache.hadoop.tools.DistCp$CopyFilesMapper.copy(DistCp.java:435)
at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:543)
at org.apache.hadoop.tools.DistCp$CopyFilesMapper.map(DistCp.java:310)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:358)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:159)

This means that read itself didn't fail but the resulted file was somehow smaller.

Issue Links

• incorporates: HDFS-1303 StreamFile.doGet(..) uses an additional RPC to get file length
• is related to: HDFS-1024 SecondaryNamenode fails to checkpoint because namenode fails with CancelledKeyException
• is related to: HDFS-3318 Hftp hangs on transfers >2GB
• relates to: HDFS-1102 HftpFileSystem : errors during transfer result in truncated transfer
• relates to: HDFS-1304 There is no unit test for HftpFileSystem.open(..)