[HDFS-5761] DataNode fails to validate integrity for checksum type NULL when DataNode recovers

• Type: Bug
• Status: Patch Available
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.0.0-alpha1
• Component/s: datanode
• Labels: BB2015-05-TBR
• Assignee: Kousuke Saruta
• Reporter: Kousuke Saruta
• Created: 13/Jan/14 16:06
• Updated: 12/May/16 18:15

Description

When DataNode is down during writing blocks, the blocks are not filinalized and the next time DataNode recovers, integrity validation will run.
But if we use NULL for checksum algorithm (we can set NULL to dfs.checksum.type), DataNode will fail to validate integrity and cannot be up.

The cause is in BlockPoolSlice#validateIntegrity.
In the method, there is following code.

long numChunks = Math.min(
          (blockFileLen + bytesPerChecksum - 1)/bytesPerChecksum, 
          (metaFileLen - crcHeaderLen)/checksumSize);

When we choose NULL checksum, checksumSize is 0 so ArithmeticException will be thrown and DataNode cannot be up.