Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-13682

Cannot create encryption zone after KMS auth token expires

    XMLWordPrintableJSON

Details

    Description

      Our internal testing reported this behavior recently.

      [root@nightly6x-1 ~]# sudo -u hdfs /usr/bin/kinit -kt /cdep/keytabs/hdfs.keytab hdfs -l 30d -r 30d
      [root@nightly6x-1 ~]# sudo -u hdfs klist
      Ticket cache: FILE:/tmp/krb5cc_994
      Default principal: hdfs@GCE.CLOUDERA.COM
      
      Valid starting       Expires              Service principal
      06/12/2018 03:24:09  07/12/2018 03:24:09  krbtgt/GCE.CLOUDERA.COM@GCE.CLOUDERA.COM
      [root@nightly6x-1 ~]# sudo -u hdfs hdfs crypto -createZone -keyName key77 -path /user/systest/ez
      RemoteException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Failed to find any Kerberos tgt)
      

      Upon further investigation, it's due to the KMS client (cached in HDFS NN) cannot authenticate with the server after the authentication token (which is cached by KMSCP) expires, even if the HDFS client RPC has valid kerberos credentials.

      Attachments

        1. HDFS-13682.01.patch
          5 kB
          Xiao Chen
        2. HDFS-13682.02.patch
          6 kB
          Xiao Chen
        3. HDFS-13682.03.patch
          6 kB
          Xiao Chen
        4. HDFS-13682.dirty.repro.branch-2.patch
          16 kB
          Xiao Chen
        5. HDFS-13682.dirty.repro.patch
          15 kB
          Xiao Chen

        Issue Links

          Activity

            People

              xiaochen Xiao Chen
              xiaochen Xiao Chen
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: