Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13682

Cannot create encryption zone after KMS auth token expires

    Details

      Description

      Our internal testing reported this behavior recently.

      [root@nightly6x-1 ~]# sudo -u hdfs /usr/bin/kinit -kt /cdep/keytabs/hdfs.keytab hdfs -l 30d -r 30d
      [root@nightly6x-1 ~]# sudo -u hdfs klist
      Ticket cache: FILE:/tmp/krb5cc_994
      Default principal: hdfs@GCE.CLOUDERA.COM
      
      Valid starting       Expires              Service principal
      06/12/2018 03:24:09  07/12/2018 03:24:09  krbtgt/GCE.CLOUDERA.COM@GCE.CLOUDERA.COM
      [root@nightly6x-1 ~]# sudo -u hdfs hdfs crypto -createZone -keyName key77 -path /user/systest/ez
      RemoteException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      

      Upon further investigation, it's due to the KMS client (cached in HDFS NN) cannot authenticate with the server after the authentication token (which is cached by KMSCP) expires, even if the HDFS client RPC has valid kerberos credentials.

        Attachments

        1. HDFS-13682.dirty.repro.patch
          15 kB
          Xiao Chen
        2. HDFS-13682.dirty.repro.branch-2.patch
          16 kB
          Xiao Chen
        3. HDFS-13682.03.patch
          6 kB
          Xiao Chen
        4. HDFS-13682.02.patch
          6 kB
          Xiao Chen
        5. HDFS-13682.01.patch
          5 kB
          Xiao Chen

          Activity

            People

            • Assignee:
              xiaochen Xiao Chen
              Reporter:
              xiaochen Xiao Chen
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: