Uploaded image for project: 'Atlas'
  1. Atlas
  2. ATLAS-354

Kerberized cluster: quick_start.py fails to add sample data

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 0.6-incubating
    • 0.6-incubating
    • None
    • None

    Description

      Check the ticket cache available

      [atlas@os-r7-apathan-hbase-1 ~]$ klist
      Ticket cache: FILE:/tmp/krb5cc_1010
      Default principal: atlas/os-r7-apathan-hbase-1.novalocal@EXAMPLE.COM
      
      Valid starting       Expires              Service principal
      12/01/2015 17:57:14  12/02/2015 17:57:14  krbtgt/EXAMPLE.COM@EXAMPLE.COM
      [atlas@os-r7-apathan-hbase-1 ~]$
      

      Below is the client.properties from kerberized cluster

      #########  Security Properties  #########
      
      # SSL config
      
      atlas.enableTLS=false
      truststore.file=/path/to/truststore.jks
      cert.stores.credential.provider.path=jceks://file/path/to/credentialstore.jceks
      # following only required for 2-way SSL
      keystore.file=/path/to/keystore.jks
      
      # Authentication config
      
      # enabled:  true or false
      atlas.http.authentication.enabled=false
      # type:  simple or kerberos
      atlas.http.authentication.type=simple
      #########  Security Properties  #########
      

      Now try running quick_start.py, it throws below exception

      Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: Authentication failed, status: 401, message: Authentication required
      	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
      	at com.sun.jersey.api.client.Client.handle(Client.java:648)
      	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
      	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
      	at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
      	at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
      	at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:370)
      	at org.apache.atlas.AtlasClient.createType(AtlasClient.java:170)
      	at org.apache.atlas.examples.QuickStart.createTypes(QuickStart.java:97)
      	at org.apache.atlas.examples.QuickStart.main(QuickStart.java:57)
      Caused by: java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: Authentication failed, status: 401, message: Authentication required
      	at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
      	at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
      	at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
      	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
      	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
      	... 9 more
      Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: Authentication failed, status: 401, message: Authentication required
      	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:274)
      	at org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:77)
      	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
      	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
      	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
      	at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
      	... 16 more
      Example data added to Apache Atlas Server!!!
      

      To fix the above issue, I have tried adding authentication config to client.properties manually, like below

      atlas.enableTLS=false
      truststore.file=/path/to/truststore.jks
      cert.stores.credential.provider.path=jceks://file/path/to/credentialstore.jceks
      # following only required for 2-way SSL
      keystore.file=/path/to/keystore.jks
      
      # Authentication config
      
      # enabled:  true or false
      atlas.http.authentication.enabled=true
      # type:  simple or kerberos
      atlas.http.authentication.type=kerberos
      #########  Security Properties  #########
      
      
      atlas.authentication.keytab=/etc/security/keytabs/atlas.service.keytab
      atlas.authentication.method=kerberos
      atlas.authentication.principal=atlas/_HOST@EXAMPLE.COM
      atlas.http.authentication.kerberos.keytab=/etc/security/keytabs/spnego.service.keytab
      atlas.http.authentication.kerberos.name.rules=RULE:[1:$1@$0](ambari-qa@EXAMPLE.COM)s/.*/ambari-qa/ \
      RULE:[1:$1@$0](hbase@EXAMPLE.COM)s/.*/hbase/ \
      RULE:[1:$1@$0](hdfs@EXAMPLE.COM)s/.*/hdfs/ \
      RULE:[1:$1@$0](.*@EXAMPLE.COM)s/@.*// \
      RULE:[2:$1@$0](amshbase@EXAMPLE.COM)s/.*/ams/ \
      RULE:[2:$1@$0](amszk@EXAMPLE.COM)s/.*/ams/ \
      RULE:[2:$1@$0](atlas@EXAMPLE.COM)s/.*/atlas/ \
      RULE:[2:$1@$0](dn@EXAMPLE.COM)s/.*/hdfs/ \
      RULE:[2:$1@$0](hbase@EXAMPLE.COM)s/.*/hbase/ \
      RULE:[2:$1@$0](hive@EXAMPLE.COM)s/.*/hive/ \
      RULE:[2:$1@$0](jhs@EXAMPLE.COM)s/.*/mapred/ \
      RULE:[2:$1@$0](jn@EXAMPLE.COM)s/.*/hdfs/ \
      RULE:[2:$1@$0](nfs@EXAMPLE.COM)s/.*/hdfs/ \
      RULE:[2:$1@$0](nm@EXAMPLE.COM)s/.*/yarn/ \
      RULE:[2:$1@$0](nn@EXAMPLE.COM)s/.*/hdfs/ \
      RULE:[2:$1@$0](rm@EXAMPLE.COM)s/.*/yarn/ \
      RULE:[2:$1@$0](yarn@EXAMPLE.COM)s/.*/yarn/ \
      DEFAULT
      atlas.http.authentication.kerberos.principal=HTTP/_HOST@EXAMPLE.COM
      

      with the new auth config, tried running /grid/0/hdp/current/atlas-server/bin/quick_start.py, exception with "Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER"

      Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)
      	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
      	at com.sun.jersey.api.client.Client.handle(Client.java:648)
      	at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
      	at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
      	at com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
      	at org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
      	at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:370)
      	at org.apache.atlas.AtlasClient.createType(AtlasClient.java:170)
      	at org.apache.atlas.examples.QuickStart.createTypes(QuickStart.java:97)
      	at org.apache.atlas.examples.QuickStart.main(QuickStart.java:57)
      Caused by: java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)
      	at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
      	at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
      	at org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
      	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
      	at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
      	... 9 more
      Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:332)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:205)
      	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
      	at org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
      	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticatedURL.openConnection(DelegationTokenAuthenticatedURL.java:322)
      	at org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:103)
      	... 16 more
      Caused by: GSSException: No valid credentials provided (Mechanism level: Server not found in Kerberos database (7) - LOOKING_UP_SERVER)
      	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:710)
      	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:248)
      	at sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:311)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator$1.run(KerberosAuthenticator.java:287)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.authentication.client.KerberosAuthenticator.doSpnegoSequence(KerberosAuthenticator.java:287)
      	... 21 more
      Caused by: KrbException: Server not found in Kerberos database (7) - LOOKING_UP_SERVER
      	at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:73)
      	at sun.security.krb5.KrbTgsReq.getReply(KrbTgsReq.java:191)
      	at sun.security.krb5.KrbTgsReq.sendAndGetCreds(KrbTgsReq.java:202)
      	at sun.security.krb5.internal.CredentialsUtil.serviceCreds(CredentialsUtil.java:292)
      	at sun.security.krb5.internal.CredentialsUtil.acquireServiceCreds(CredentialsUtil.java:101)
      	at sun.security.krb5.Credentials.acquireServiceCreds(Credentials.java:456)
      	at sun.security.jgss.krb5.Krb5Context.initSecContext(Krb5Context.java:641)
      	... 28 more
      Caused by: KrbException: Identifier doesn't match expected value (906)
      	at sun.security.krb5.internal.KDCRep.init(KDCRep.java:140)
      	at sun.security.krb5.internal.TGSRep.init(TGSRep.java:65)
      	at sun.security.krb5.internal.TGSRep.<init>(TGSRep.java:60)
      	at sun.security.krb5.KrbTgsRep.<init>(KrbTgsRep.java:55)
      	... 34 more
      

      Attachments

        1. ATLAS-354.patch
          4 kB
          Shwetha GS
        2. ATLAS-354-v2.patch
          10 kB
          Shwetha GS
        3. ATLAS-354-v3.patch
          11 kB
          Shwetha GS

        Activity

          People

            shwethags Shwetha GS
            ayubpathan Ayub Pathan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: