Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-5853

Hive Lock Manager leaks zookeeper connections

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.10.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Hive 0.10 leaks zookeeper connections from ZooKeeperHiveLockManager. HIVE-3723 describes a similar issue for cases of semantic errors and failures, but we're experiencing a consistent connection leak per query (even simple successful queries like "select * from dual").

      Workaround: When turning off hive.support.concurrency, everything works fine - no leak (obviously, since the lock manager is not used).

      Details:

      OS: CentOS 5.9
      Hive version: hive-server-0.10.0+67-1.cdh4.2.0.p0.10.el5 and hive-0.10.0+198-1.cdh4.4.0.p0.15.el5
      Hadoop version: CDH4.2
      Namenode uses HA. Hive's zookeeper configuration uses the NN zookeeper.

      The problem occurs both when using the python thrift API, and the java thrift API.

      The leak happens even when we're running repeated "select * from dual" queries. We've checked the zookeeper connections using "netstat -n | grep 2181 | grep ESTAB | wc -l".

      Eventually, the connection from the client reach the max connections per client limit in ZK, causing new queries to get stuck and never return.

      We'll gladly provide more information if needed.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                harelba Harel Ben Attia
              • Votes:
                1 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated: