Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-13185

Multiple Instances of KMS Behind a Load-Balancer or VIP AuthenticationException

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.6.1
    • None
    • None
    • None

    Description

      hive MR:

      Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: org.apache.hadoop.security.token.SecretManager$InvalidToken: token (owner=hive, renewer=yarn, realUser=, issueDate=1463650744837, maxDate=1464255544837, sequenceNumber=42, masterKeyId=2) can't be found in cache
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
      at org.apache.hadoop.util.HttpExceptionUtils.validateResponse(HttpExceptionUtils.java:157)
      at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:487)
      at org.apache.hadoop.crypto.key.kms.KMSClientProvider.call(KMSClientProvider.java:445)
      at org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:719)
      at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:388)
      at org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(DFSClient.java:1347)
      at org.apache.hadoop.hdfs.DFSClient.createWrappedInputStream(DFSClient.java:1414)
      at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:304)
      at org.apache.hadoop.hdfs.DistributedFileSystem$3.doCall(DistributedFileSystem.java:298)
      at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:298)
      at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:766)
      at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:108)
      at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
      at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:250)
      ... 9 more

      KMS_LOG:
      403

      Attachments

        Activity

          People

            Unassigned Unassigned
            lushuai lushuai
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: