Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-1644

Add support for compressed SetWatches packet

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: c client, java client, server
    • Labels:
      None

      Description

      On reconnect with a server to restore its session, a client have to send all watched paths via SetWatches packet to the server. This can be potentially large and exceeded server-side buffer (jute.maxbuffer) causing the session to fail. We have 2 concerns.

      1. We can increase jute.maxbuffer to arbitrarily size as a simple workaround, but, in our use case, the number of watch is going to keep growing

      2. If a large number of clients get disconnected at once, the server may receive a large amount data over network because of the flood of SetWatches packet.

      In our case, the watch paths should by highly compressible. So our current plan is to add a new type of request which is a compressed set watch request. It should be possible to support multiple compression schemes. We are probably going to use snappy compression but may add support for gzip as a default to minimize external dependency requirement.

      Feel free to comment if you have any suggestion.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                thawan Thawan Kooburat
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: