[HDFS-12931] Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.1.0
• Component/s: encryption
• Hadoop Flags: Reviewed
• Assignee: Mukul Kumar Singh
• Reporter: Mukul Kumar Singh
• Created: 15/Dec/17 10:22
• Updated: 03/Jan/18 23:22
• Resolved: 03/Jan/18 23:05

Description

Hive external table create fails because of HDFS fails to fetch block MD5 checksum.
This happens because of the InvalidEncryptionKeyException.

org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException: Can't re-compute encryption key for nonce, since the required block key (keyID=-1675775329) doesn't exist. Current key: 1496422662
	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiatedCipherOption(DataTransferSaslUtil.java:417)
	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:478)
	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getEncryptedStreams(SaslDataTransferClient.java:300)
	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:241)
	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.newSocketSend(SaslDataTransferClient.java:142)
	at org.apache.hadoop.hdfs.DFSClient.connectToDN(DFSClient.java:2450)
	at org.apache.hadoop.hdfs.DFSClient.getFileChecksum(DFSClient.java:2310)
	at org.apache.hadoop.hdfs.DistributedFileSystem$30.doCall(DistributedFileSystem.java:1569)
	at org.apache.hadoop.hdfs.DistributedFileSystem$30.doCall(DistributedFileSystem.java:1565)
	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:1577)