Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12931

Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.1.0
    • encryption
    • None
    • Reviewed

    Description

      Hive external table create fails because of HDFS fails to fetch block MD5 checksum.
      This happens because of the InvalidEncryptionKeyException.

      org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException: Can't re-compute encryption key for nonce, since the required block key (keyID=-1675775329) doesn't exist. Current key: 1496422662
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiatedCipherOption(DataTransferSaslUtil.java:417)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:478)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getEncryptedStreams(SaslDataTransferClient.java:300)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:241)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.newSocketSend(SaslDataTransferClient.java:142)
      	at org.apache.hadoop.hdfs.DFSClient.connectToDN(DFSClient.java:2450)
      	at org.apache.hadoop.hdfs.DFSClient.getFileChecksum(DFSClient.java:2310)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$30.doCall(DistributedFileSystem.java:1569)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$30.doCall(DistributedFileSystem.java:1565)
      	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:1577)
      

      Attachments

        1. HDFS-12931.001.patch
          2 kB
          Mukul Kumar Singh
        2. HDFS-12931.002.patch
          5 kB
          Mukul Kumar Singh
        3. HDFS-12931.003.patch
          5 kB
          Mukul Kumar Singh

        Activity

          People

            msingh Mukul Kumar Singh
            msingh Mukul Kumar Singh
            Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: