[ZOOKEEPER-1644] Add support for compressed SetWatches packet

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: c client, java client, server
• Assignee: Unassigned
• Reporter: Thawan Kooburat
• Created: 11/Feb/13 22:13
• Updated: 16/Feb/13 00:42

Description

On reconnect with a server to restore its session, a client have to send all watched paths via SetWatches packet to the server. This can be potentially large and exceeded server-side buffer (jute.maxbuffer) causing the session to fail. We have 2 concerns.

1. We can increase jute.maxbuffer to arbitrarily size as a simple workaround, but, in our use case, the number of watch is going to keep growing

2. If a large number of clients get disconnected at once, the server may receive a large amount data over network because of the flood of SetWatches packet.

In our case, the watch paths should by highly compressible. So our current plan is to add a new type of request which is a compressed set watch request. It should be possible to support multiple compression schemes. We are probably going to use snappy compression but may add support for gzip as a default to minimize external dependency requirement.

Feel free to comment if you have any suggestion.

Issue Links

• relates to: ZOOKEEPER-706 large numbers of watches can cause session re-establishment to fail