[SPARK-32570] Thriftserver LDAP failed

• Type: Question
• Status: Resolved
• Priority: Major
• Resolution: Invalid
• Affects Version/s: 2.4.6
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Jie Zhang
• Created: 07/Aug/20 21:47
• Updated: 16/Mar/23 04:44
• Resolved: 08/Aug/20 02:49

Description

I downloaded spark-2.4.6-bin-hadoop2.7.tgz, added a new file to conf/hive-site.xml, put the following parameters into it, ran sbin/start-thriftserver.sh, then bin/beeline worked, able to query tables in our hive-metastore. 

<property>
    <name>hive.metastore.uris</name>
    <value>thrift://hive-metastore-service.company.com:9083</value>
</property>

<property>
    <name>hive.metastore.schema.verification</name>
    <value>false</value>
</property>

<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://hive-metastore-db.company.com:3306/hive?createDatabaseIfNotExist=false</value>
</property>

<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>org.mariadb.jdbc.Driver</value>
</property>

<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>xxxxxxxxx</value>
</property>  

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>xxxxxxxxx</value>
</property>  

<property>
    <name>hive.metastore.connect.retries</name>
    <value>15</value>
</property>

In order to enable LDAP, I added these parameters into conf/hive-site.xml, stopped and started thriftserver, then bin/beeline complained invalid credentials.

I know my credentials works because I enabled LDAP on Hive-Server2 and it worked. 

<property>
    <name>hive.server2.authentication</name>
    <value>LDAP</value>
</property>  

<property>
    <name>hive.server2.authentication.ldap.url</name>
    <value>ldaps://ldap-server.company.com:636</value>
</property>  

<property>
    <name>hive.server2.authentication.ldap.baseDN</name>
    <value>ou=People,dc=company,dc=com</value>
</property>  

<property>
    <name>hive.server2.authentication.ldap.userDNPattern</name>
    <value>cn=%s,ou=People,dc=company,dc=com</value>
</property>

The error message:

20/08/07 21:05:39 ERROR TSaslTransport: SASL negotiation failure20/08/07 21:05:39 ERROR TSaslTransport: SASL negotiation failurejavax.security.sasl.SaslException: Error validating the login [Caused by javax.security.sasl.AuthenticationException: Error validating LDAP user [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]]] at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:109) at org.apache.thrift.transport.TSaslTransport$SaslParticipant.evaluateChallengeOrResponse(TSaslTransport.java:539) at org.apache.thrift.transport.TSaslTransport.open(TSaslTransport.java:283) at org.apache.thrift.transport.TSaslServerTransport.open(TSaslServerTransport.java:41) at org.apache.thrift.transport.TSaslServerTransport$Factory.getTransport(TSaslServerTransport.java:216) at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:269) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)Caused by: javax.security.sasl.AuthenticationException: Error validating LDAP user [Caused by javax.naming.AuthenticationException: [LDAP: error code 49 - Invalid Credentials]] at org.apache.hive.service.auth.LdapAuthenticationProviderImpl.Authenticate(LdapAuthenticationProviderImpl.java:77) at org.apache.hive.service.auth.PlainSaslHelper$PlainServerCallbackHandler.handle(PlainSaslHelper.java:106) at org.apache.hive.service.auth.PlainSaslServer.evaluateResponse(PlainSaslServer.java:102) ... 8 more

Anything else I need to do in order to enable LDAP on Spark Thriftserver? Thanks for your help.