[HDFS-5843] DFSClient.getFileChecksum() throws IOException if checksum is disabled

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.4.0
• Component/s: datanode
• Hadoop Flags: Reviewed
• Assignee: Laurent Goujon
• Reporter: Laurent Goujon
• Created: 28/Jan/14 06:05
• Updated: 10/Apr/14 13:12
• Resolved: 30/Jan/14 19:24

Description

If a file is created with checksum disabled (using ChecksumOpt.disabled() for example), calling FileSystem.getFileChecksum() throws the following IOException:

java.io.IOException: Fail to get block MD5 for BP-341493254-192.168.1.10-1390888724459:blk_1073741825_1001
	at org.apache.hadoop.hdfs.DFSClient.getFileChecksum(DFSClient.java:1965)
	at org.apache.hadoop.hdfs.DFSClient.getFileChecksum(DFSClient.java:1771)
	at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1186)
	at org.apache.hadoop.hdfs.DistributedFileSystem$21.doCall(DistributedFileSystem.java:1)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:1194)
[...]

From the logs, the datanode is doing some wrong arithmetics because of the crcPerBlock:

2014-01-27 21:58:46,329 ERROR datanode.DataNode (DataXceiver.java:run(225)) - 127.0.0.1:52398:DataXceiver error processing BLOCK_CHECKSUM operation  src: /127.0.0.1:52407 dest: /127.0.0.1:52398
java.lang.ArithmeticException: / by zero
	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.blockChecksum(DataXceiver.java:658)
	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opBlockChecksum(Receiver.java:169)
	at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:77)
	at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:221)
	at java.lang.Thread.run(Thread.java:695)