Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
2.5.0
-
None
-
None
Description
Hello,
I'm working on an hadoop 3.3.6 stack with Hive 3.1.3, tez 0.10.2 and spark 3.5.1.
The issue is ranger cannot connect to my hive service. I have those errors :
hiveserver2.out :
12:16:43.952 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - <== RangerPluginClassLoader.findResources(META-INF/services/javax.ws.rs.ext.MessageBodyWriter) 12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - ==> RangerPluginClassLoader.loadClass(javax.mail.MessagingException) 12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - RangerPluginClassLoader.loadClass(javax.mail.MessagingException): calling childClassLoader.findClass() 12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - ==> RangerPluginClassLoader.findClass(javax.mail.MessagingException) 12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - RangerPluginClassLoader.findClass(javax.mail.MessagingException): calling childClassLoader().findClass() 12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - RangerPluginClassLoader.findClass(javax.mail.MessagingException): calling componentClassLoader.findClass() 12:16:43.957 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.classloader.RangerPluginClassLoader - RangerPluginClassLoader.loadClass(javax.mail.MessagingException): calling componentClassLoader.loadClass() 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] ERROR org.apache.ranger.admin.client.RangerAdminRESTClient - Failed to get response, Error is : null 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - <== RangerAdminRESTClient.getRangerRolesDownloadResponse(-1, 1728128759797): null 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] ERROR org.apache.ranger.admin.client.RangerAdminRESTClient - Error getting Roles; Received NULL response!!. secureMode=true, user=hive/master-03.tdp@REALM.TDP (auth:KERBEROS), serviceName=hive-tdp 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - <== RangerAdminRESTClient.getRolesIfUpdatedWithCred(-1, 1728128759797): null 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - <== RangerAdminRESTClient.getRolesIfUpdated(-1, 1728128759797): 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.util.RangerRolesProvider - RangerRolesProvider(serviceName=hive-tdp).run(): no update found. lastKnownRoleVersion=-1 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.perf.policyengine.init - [PERF]:PolicyRefresher(serviceName=hive-tdp)-36:RangerRolesProvider.loadUserGroupRolesFromAdmin(serviceName=hive-tdp):177226260:214664927 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.util.RangerRolesProvider - <== RangerRolesProvider(serviceName=hive-tdp serviceType= hive ).loadUserGroupRolesFromAdmin() 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.perf.policyengine.init - In-Use memory: 307461672, Free memory:71074264 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.perf.policyengine.init - [PERF]:PolicyRefresher(serviceName=hive-tdp)-36:RangerRolesProvider.loadUserGroupRoles(serviceName=hive-tdp):177292518:214730011 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.util.RangerRolesProvider - <== RangerRolesProvider(serviceName=hive-tdp).loadUserGroupRoles() 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.util.PolicyRefresher - <== PolicyRefresher(serviceName=hive-tdp).loadRoles() 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.util.PolicyRefresher - ==> PolicyRefresher(serviceName=hive-tdp).loadPolicy() 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.perf.policyengine.init - In-Use memory: 307461672, Free memory:71074264 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.plugin.util.PolicyRefresher - ==> PolicyRefresher(serviceName=hive-tdp).loadPolicyfromPolicyAdmin() 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - ==> RangerAdminRESTClient.getServicePoliciesIfUpdated(-1, 1728128790697) 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - ==> RangerAdminRESTClient.getServicePoliciesIfUpdatedWithCred(-1, 1728128790697) 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - ==> RangerAdminRESTClient.getRangerAdminPolicyDownloadResponse(-1, 1728128790697) 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.ranger.admin.client.RangerAdminRESTClient - Checking Service policy if updated as user : hive/master-03.tdp@REALM.TDP (auth:KERBEROS) 12:16:43.963 [PolicyRefresher(serviceName=hive-tdp)-36] DEBUG org.apache.hadoop.security.UserGroupInformation - PrivilegedAction [as: hive/master-03.tdp@REALM.TDP (auth:KERBEROS)][action: org.apache.ranger.admin.client.RangerAdminRESTClient$13@ab6302e] java.lang.Exception: null at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1873) at org.apache.ranger.admin.client.RangerAdminRESTClient.getRangerAdminPolicyDownloadResponse(RangerAdminRESTClient.java:948) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdatedWithCred(RangerAdminRESTClient.java:823) at org.apache.ranger.admin.client.RangerAdminRESTClient.getServicePoliciesIfUpdated(RangerAdminRESTClient.java:127) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicyfromPolicyAdmin(PolicyRefresher.java:302) at org.apache.ranger.plugin.util.PolicyRefresher.loadPolicy(PolicyRefresher.java:241) at org.apache.ranger.plugin.util.PolicyRefresher.run(PolicyRefresher.java:203)
Test connection on the ranger admin UI : 
My ranger logs :
2024-10-05 12:06:45,082 [https-jsse-nio-6182-exec-13] INFO [RangerTransactionService.java:156] RangerTransactionService: tasks(scheduled=52, executed=51, failed=0, pending=1) 2024-10-05 12:06:47,733 [https-jsse-nio-6182-exec-9] INFO [TagDBStore.java:1379] SUPPORTS_TAG_DELTAS=false 2024-10-05 12:06:47,734 [https-jsse-nio-6182-exec-9] INFO [TagDBStore.java:1380] SUPPORTS_IN_PLACE_TAG_UPDATES=false 2024-10-05 12:20:44,433 [https-jsse-nio-6182-exec-6] INFO [SpringEventListener.java:76] Login Successful:admin | Ip Address:192.168.56.1 | sessionId=ED3F33CD1C50E5B70088756FA2D196C3 | Epoch=1728130844433 2024-10-05 12:20:52,264 [timed-executor-pool-0] INFO [BaseClient.java:132] Init Lookup Login: security enabled, using lookupPrincipal/lookupKeytab 2024-10-05 12:20:52,271 [timed-executor-pool-0] INFO [HiveClient.java:85] Secured Mode: JDBC Connection done with preAuthenticated Subject 2024-10-05 12:20:52,274 [timed-executor-pool-0-SendThread(master-03.tdp:2181)] WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207) 2024-10-05 12:20:52,275 [timed-executor-pool-0-EventThread] ERROR [ConnectionState.java:247] Authentication failed 2024-10-05 12:20:52,571 [timed-executor-pool-0-SendThread(master-03.tdp:2181)] WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207) 2024-10-05 12:20:52,572 [timed-executor-pool-0-EventThread] ERROR [ConnectionState.java:247] Authentication failed 2024-10-05 12:20:53,387 [timed-executor-pool-0] WARN [HiveConnection.java:237] Failed to connect to master-03:10001 2024-10-05 12:20:53,391 [timed-executor-pool-0-SendThread(master-01.tdp:2181)] WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207) 2024-10-05 12:20:53,392 [timed-executor-pool-0-EventThread] ERROR [ConnectionState.java:247] Authentication failed 2024-10-05 12:20:53,523 [timed-executor-pool-0] WARN [HiveConnection.java:264] Could not open client transport with JDBC Uri: jdbc:hive2://master-03:10001/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2;sslTrustStore=/etc/ssl/certs/truststore.jks;trustStorePassword=Truststore123!: Failed to open new session: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:No privilege 'Select' found for inputs { database:default}) Retrying 0 of 1 2024-10-05 12:20:54,903 [timed-executor-pool-0] WARN [HiveConnection.java:237] Failed to connect to master-02:10001 2024-10-05 12:20:54,908 [timed-executor-pool-0-SendThread(master-02.tdp:2181)] WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207) 2024-10-05 12:20:54,908 [timed-executor-pool-0-EventThread] ERROR [ConnectionState.java:247] Authentication failed 2024-10-05 12:20:55,035 [timed-executor-pool-0] ERROR [Utils.java:619] Unable to read HiveServer2 configs from ZooKeeper 2024-10-05 12:20:55,040 [timed-executor-pool-0-SendThread(master-03.tdp:2181)] WARN [ClientCnxn.java:1163] SASL configuration failed. Will continue connection to Zookeeper server without SASL authentication, if Zookeeper server allows it. javax.security.auth.login.LoginException: No JAAS configuration section named 'Client' was found in specified JAAS configuration file: '/dev/null'. at org.apache.zookeeper.client.ZooKeeperSaslClient.<init>(ZooKeeperSaslClient.java:190) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1157) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1207) 2024-10-05 12:20:55,041 [timed-executor-pool-0-EventThread] ERROR [ConnectionState.java:247] Authentication failed 2024-10-05 12:20:55,321 [timed-executor-pool-0] ERROR [HiveClient.java:570] Unable to Connect to Hive org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive Thrift Server instance. at org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:729) at org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:568) at org.apache.ranger.services.hive.client.HiveClient.access$000(HiveClient.java:56) at org.apache.ranger.services.hive.client.HiveClient$1.run(HiveClient.java:88) at org.apache.ranger.services.hive.client.HiveClient$1.run(HiveClient.java:86) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.ranger.services.hive.client.HiveClient.initHive(HiveClient.java:86) at org.apache.ranger.services.hive.client.HiveClient.<init>(HiveClient.java:77) at org.apache.ranger.services.hive.client.HiveClient.connectionTest(HiveClient.java:827) at org.apache.ranger.services.hive.client.HiveResourceMgr.connectionTest(HiveResourceMgr.java:49) at org.apache.ranger.services.hive.RangerServiceHive.validateConfig(RangerServiceHive.java:82) at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:732) at org.apache.ranger.biz.ServiceMgr$ValidateCallable.actualCall(ServiceMgr.java:719) at org.apache.ranger.biz.ServiceMgr$TimedCallable.call(ServiceMgr.java:680) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:750) Caused by: java.sql.SQLException: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:170) at org.apache.hive.jdbc.HiveDriver.connect(HiveDriver.java:107) at java.sql.DriverManager.getConnection(DriverManager.java:664) at java.sql.DriverManager.getConnection(DriverManager.java:270) at org.apache.ranger.services.hive.client.HiveClient.initConnection(HiveClient.java:712) ... 18 common frames omitted Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 configs from ZooKeeper at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:147) at org.apache.hive.jdbc.Utils.configureConnParamsFromZooKeeper(Utils.java:511) at org.apache.hive.jdbc.Utils.parseURL(Utils.java:334) at org.apache.hive.jdbc.HiveConnection.<init>(HiveConnection.java:168) ... 22 common frames omitted Caused by: org.apache.hive.jdbc.ZooKeeperHiveClientException: Unable to read HiveServer2 uri from ZooKeeper: at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.updateParamsWithZKServerNode(ZooKeeperHiveClientHelper.java:125) at org.apache.hive.jdbc.ZooKeeperHiveClientHelper.configureConnParams(ZooKeeperHiveClientHelper.java:145) ... 25 common frames omitted 2024-10-05 12:20:55,321 [timed-executor-pool-0] ERROR [HiveResourceMgr.java:51] <== HiveResourceMgr.connectionTest Error: org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive Thrift Server instance. 2024-10-05 12:20:55,322 [timed-executor-pool-0] ERROR [RangerServiceHive.java:84] <== RangerServiceHive.validateConfig Error:org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive Thrift Server instance. 2024-10-05 12:20:55,322 [timed-executor-pool-0] ERROR [ServiceMgr.java:682] TimedCallable.call: Error:org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive Thrift Server instance. 2024-10-05 12:20:55,322 [https-jsse-nio-6182-exec-4] ERROR [ServiceMgr.java:231] ==> ServiceMgr.validateConfig Error:org.apache.ranger.plugin.client.HadoopException: org.apache.ranger.plugin.client.HadoopException: Unable to connect to Hive Thrift Server instance.
It tells me that i have No JAAS configuration section named 'Client' but I have one configured in my zookeeper edge node.
I don't now what's the issue, if anyone can help me.