Major Description
Currently we open tons of new connections to Zookeeper, like every time we instantiate a new HTable. There is a maximum number of client connections as described here:
<property>
<name>hbase.zookeeper.property.maxClientCnxns</name>
<value>30</value>
<description>Property from ZooKeeper's config zoo.cfg.
Limit on number of concurrent connections (at the socket level) that a
single client, identified by IP address, may make to a single member of
the ZooKeeper ensemble. Set high to avoid zk connection issues running
standalone and pseudo-distributed.
</description>
</property>
If you hit that max number, ZK will just refuse your connections. Suppose you have 4 maps running on a server hosting a RS, you may actually lose your connection in the RS and eventually hit a session timeout. Maybe we should singleton ZKW?