[ACCUMULO-2757] millions of zookeeper watches

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Later
• Component/s: client, master, tserver

Environment

very large clusters

• Assignee: Eric C. Newton
• Reporter: Eric C. Newton
• Created: 29/Apr/14 18:31
• Updated: 20/Feb/17 22:14
• Resolved: 02/Jul/14 15:19

Description

I recently learned how to get the number of watches in a zookeeper node.

$ echo wchp | nc localhost 2181

There are a surprising number of watches supporting table configuration. A single tserver accumulo instance with one data table is using 872 watches. Large clusters, even with many zookeepers, can have millions of watches.

We may need to watch a single notification node, or serialize the table configs into a single node per table. That would accomplish ACCUMULO-1568, too.

Issue Links

• relates to: ACCUMULO-1568 Make configuration changes Atomic
• relates to: ACCUMULO-2841 Arbitrary namespace and table metadata tags
• relates to: ACCUMULO-3842 [UMBRELLA] Remove non-transient data from ZooKeeper

Sub-Tasks

• eliminate table watches in clients by using Thrift to find the table map: Open