Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12931

Handle InvalidEncryptionKeyException during DistributedFileSystem#getFileChecksum

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.1.0
    • Component/s: encryption
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Hive external table create fails because of HDFS fails to fetch block MD5 checksum.
      This happens because of the InvalidEncryptionKeyException.

      org.apache.hadoop.hdfs.protocol.datatransfer.InvalidEncryptionKeyException: Can't re-compute encryption key for nonce, since the required block key (keyID=-1675775329) doesn't exist. Current key: 1496422662
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.DataTransferSaslUtil.readSaslMessageAndNegotiatedCipherOption(DataTransferSaslUtil.java:417)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.doSaslHandshake(SaslDataTransferClient.java:478)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.getEncryptedStreams(SaslDataTransferClient.java:300)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.send(SaslDataTransferClient.java:241)
      	at org.apache.hadoop.hdfs.protocol.datatransfer.sasl.SaslDataTransferClient.newSocketSend(SaslDataTransferClient.java:142)
      	at org.apache.hadoop.hdfs.DFSClient.connectToDN(DFSClient.java:2450)
      	at org.apache.hadoop.hdfs.DFSClient.getFileChecksum(DFSClient.java:2310)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$30.doCall(DistributedFileSystem.java:1569)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$30.doCall(DistributedFileSystem.java:1565)
      	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.getFileChecksum(DistributedFileSystem.java:1577)
      

        Attachments

        1. HDFS-12931.001.patch
          2 kB
          Mukul Kumar Singh
        2. HDFS-12931.002.patch
          5 kB
          Mukul Kumar Singh
        3. HDFS-12931.003.patch
          5 kB
          Mukul Kumar Singh

          Activity

            People

            • Assignee:
              msingh Mukul Kumar Singh
              Reporter:
              msingh Mukul Kumar Singh
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: