Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-3715

Kerberos Authentication related tests fail for new JDK versions

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.6.0, 3.5.7
    • None

    Description

      using OpenJDK 1.8.242 or OpenJDK 11.0.6, I got some kerberos related exceptions when running the following, Kerberos Authentication related tests:
      - QuorumKerberosAuthTest
      - QuorumKerberosHostBasedAuthTest
      - SaslKerberosAuthOverSSLTest
       
      the error:

      2020-02-03 12:11:07,197 [myid:localhost:11223] - ERROR [main-SendThread(localhost:11223):ZooKeeperSaslClient@336] - An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: null (5001))]) occurred when evaluating Zookeeper Quorum Member's  received SASL token. Zookeeper Client will go to AUTH_FAILED state.
       

      more detailed stack trace:

      Found ticket for zkclient/localhost@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Tue Feb 04 13:49:14 CET 2020Found ticket for zkclient/localhost@EXAMPLE.COM to go to krbtgt/EXAMPLE.COM@EXAMPLE.COM expiring on Tue Feb 04 13:49:14 CET 2020Entered Krb5Context.initSecContext with state=STATE_NEWService ticket not found in the subject>>> Credentials serviceCredsSingle: same realmUsing builtin default etypes for default_tgs_enctypesdefault etypes for default_tgs_enctypes: 18 17 16 23.>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes128CksumType>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType>>> KrbKdcReq send: kdc=localhost TCP:62653, timeout=30000, number of retries =3, #bytes=586>>> KDCCommunication: kdc=localhost TCP:62653, timeout=30000,Attempt =1, #bytes=586>>>DEBUG: TCPClient reading 112 bytes>>> KrbKdcReq send: #bytes read=112>>> KdcAccessibility: remove localhost:62653>>> KDCRep: init() encoding tag is 126 req type is 13>>>KRBError: sTime is Mon Feb 03 13:49:14 CET 2020 1580734154000 suSec is 100 error code is 5001 error Message is null crealm is EXAMPLE.COM sname is zkquorum/localhost@EXAMPLE.COM msgType is 30>>> Credentials serviceCredsSingle: same realmUsing builtin default etypes for default_tgs_enctypesdefault etypes for default_tgs_enctypes: 18 17 16 23.>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType>>> CksumType: sun.security.krb5.internal.crypto.HmacSha1Aes128CksumType>>> EType: sun.security.krb5.internal.crypto.Aes128CtsHmacSha1EType>>> KrbKdcReq send: kdc=localhost TCP:62653, timeout=30000, number of retries =3, #bytes=586>>> KDCCommunication: kdc=localhost TCP:62653, timeout=30000,Attempt =1, #bytes=586>>>DEBUG: TCPClient reading 112 bytes>>> KrbKdcReq send: #bytes read=112>>> KdcAccessibility: remove localhost:62653>>> KDCRep: init() encoding tag is 126 req type is 13>>>KRBError: sTime is Mon Feb 03 13:49:14 CET 2020 1580734154000 suSec is 100 error code is 5001 error Message is null crealm is EXAMPLE.COM sname is zkquorum/localhost@EXAMPLE.COM msgType is 30KrbException: null (5001) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:70) at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:226) at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:237) at sun.security.krb5.internal.CredentialsUtil.serviceCredsSingle(CredentialsUtil.java:400) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:287) at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:263) at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:118) at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:490) at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:695) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248) at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179) at com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:192) at org.apache.zookeeper.client.ZooKeeperSaslClient$1.run(ZooKeeperSaslClient.java:320) at org.apache.zookeeper.client.ZooKeeperSaslClient$1.run(ZooKeeperSaslClient.java:317) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:317) at org.apache.zookeeper.client.ZooKeeperSaslClient.createSaslToken(ZooKeeperSaslClient.java:303) at org.apache.zookeeper.client.ZooKeeperSaslClient.sendSaslPacket(ZooKeeperSaslClient.java:366) at org.apache.zookeeper.client.ZooKeeperSaslClient.initialize(ZooKeeperSaslClient.java:403) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1192)Caused by: KrbException: Identifier doesn't match expected value (906) at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140) at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65) at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60) at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55) ... 20 more2020-02-03 13:49:14,942 [myid:localhost:11223] - ERROR [main-SendThread(localhost:11223):ZooKeeperSaslClient@336] - An error: (java.security.PrivilegedActionException: javax.security.sasl.SaslException: GSS initiate failed [Caused by GSSException: No valid credentials provided (Mechanism level: null (5001))]) occurred when evaluating Zookeeper Quorum Member's  received SASL token. Zookeeper Client will go to AUTH_FAILED state.
      

       
      After trying this with different JDK versions, we see that the problem seems to appear

      • between OpenJDK 8.232 and 8.242 for java 8
      • and between 11.0.3 and 11.0.6 for java 11

      There are a lot of kerberos related changes after 8.232: see https://hg.openjdk.java.net/jdk8u/jdk8u/jdk

       

      Attachments

        Activity

          People

            symat Mate Szalay-Beko
            symat Mate Szalay-Beko
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 1.5h
                1.5h