[HIVE-4132] Connection to Zookeeper is created for each client of HS2

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: HiveServer2, Server Infrastructure
• Labels: HiveServer2
• Assignee: Ferdinand Xu
• Reporter: Brock Noland
• Created: 06/Mar/13 19:53
• Updated: 04/Dec/14 05:21

Description

Currently when concurrency is enabled and HS2 is used each client will have a connection to Zookeeper. Zookeeper has a property, maxClientCnxns, in the zoo.cfg which limits the number of connections per host to a low number like 50. The stack trace of the threads accessing zookeeper is below for reference.

- org.apache.hadoop.hive.ql.lockmgr.zookeeper.ZooKeeperHiveLockManager.lock(java.util.List, boolean) @bci=94, line=220 (Interpreted frame)
 - org.apache.hadoop.hive.ql.Driver.acquireReadWriteLocks() @bci=428, line=834 (Interpreted frame)
 - org.apache.hadoop.hive.ql.Driver.run(java.lang.String) @bci=453, line=944 (Interpreted frame)
 - org.apache.hive.service.cli.operation.SQLOperation.run() @bci=126, line=95 (Interpreted frame)
 - org.apache.hive.service.cli.session.HiveSessionImpl.executeStatement(java.lang.String, java.util.Map) @bci=19, line=168 (Interpreted frame)
 - org.apache.hive.service.cli.CLIService.executeStatement(org.apache.hive.service.cli.SessionHandle, java.lang.String, java.util.Map) @bci=10, line=153 (Interpreted frame)
 - org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(org.apache.hive.service.cli.thrift.TExecuteStatementReq) @bci=41, line=190 (Interpreted frame)
 - org.apache.hive.service.cli.thrift.TCLIService$Processor$ExecuteStatement.getResult(org.apache.hive.service.cli.thrift.TCLIService$Iface, org.apache.hive.service.cli.thrift.TCLIService$ExecuteStatement_args) @bci=14, line=1133 (Interpreted frame)

Issue Links

• duplicates: HIVE-8135 Pool zookeeper connections
• is related to: HIVE-8596 HiveServer2 dynamic service discovery: ZK throws too many connections error
• is related to: HIVE-4719 EmbeddedLockManager should be shared to all clients