ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1355

Add zk.updateServerList(newServerList)

    Details

    • Type: New Feature New Feature
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.5.0
    • Component/s: c client, java client
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When the set of servers changes, we would like to update the server list stored by clients without restarting the clients.
      Moreover, assuming that the number of clients per server is the same (in expectation) in the old configuration (as guaranteed by the current list shuffling for example), we would like to re-balance client connections across the new set of servers in a way that a) the number of clients per server is the same for all servers (in expectation) and b) there is no excessive/unnecessary client migration.

      It is simple to achieve (a) without (b) - just re-shuffle the new list of servers at every client. But this would create unnecessary migration, which we'd like to avoid.

      We propose a simple probabilistic migration scheme that achieves (a) and (b) - each client locally decides whether and where to migrate when the list of servers changes. The attached document describes the scheme and shows an evaluation of it in Zookeeper. We also implemented re-balancing through a consistent-hashing scheme and show a comparison. We derived the probabilistic migration rules from a simple formula that we can also provide, if someone's interested in the proof.

      1. ZOOKEEPER-1355-13-Nov-ver2.patch
        121 kB
        Marshall McMullen
      2. ZOOKEEPER-1355-13-Nov.patch
        121 kB
        Marshall McMullen
      3. ZOOKEEPER-1355-6-Nov-ver2.patch
        120 kB
        Marshall McMullen
      4. ZOOKEEPER-1355-6-Nov.patch
        120 kB
        Marshall McMullen
      5. ZOOKEEPER-1355-13-Oct.patch
        119 kB
        Marshall McMullen
      6. ZOOKEEPER-1355-12-Oct.patch
        120 kB
        Marshall McMullen
      7. ZOOKEEPER-1355-10-Oct.patch
        89 kB
        Alexander Shraer
      8. ZOOKEEPER-1355-ver14.patch
        119 kB
        Marshall McMullen
      9. ZOOKEEPER-1355-ver13.patch
        119 kB
        Marshall McMullen
      10. ZOOKEEPER-1355-ver12-4.patch
        119 kB
        Marshall McMullen
      11. ZOOKEEPER-1355-ver12-2.patch
        120 kB
        Marshall McMullen
      12. ZOOKEEPER-1355-ver12-1.patch
        120 kB
        Marshall McMullen
      13. ZOOKEEPER-1355-ver12.patch
        121 kB
        Marshall McMullen
      14. ZOOKEEPER-1355-ver11-1.patch
        121 kB
        Marshall McMullen
      15. ZOOKEEPER-1355-ver11.patch
        121 kB
        Marshall McMullen
      16. ZOOKEEPER-1355-ver10-4.patch
        121 kB
        Marshall McMullen
      17. ZOOKEEPER-1355-ver10-4.patch
        114 kB
        Marshall McMullen
      18. ZOOKEEPER-1355-ver10-3.patch
        241 kB
        Marshall McMullen
      19. ZOOKEEPER-1355-ver10-2.patch
        114 kB
        Marshall McMullen
      20. ZOOKEEPER-1355-ver10-1.patch
        114 kB
        Marshall McMullen
      21. ZOOKEEPER-1355-ver10.patch
        113 kB
        Marshall McMullen
      22. ZOOKEEPER-1355-ver9-1.patch
        34 kB
        Alexander Shraer
      23. ZOOKEEPER-1355-ver9.patch
        34 kB
        Alexander Shraer
      24. ZOOKEEPER-1355-ver8.patch
        33 kB
        Alexander Shraer
      25. ZOOKEEPER-1355-ver7.patch
        32 kB
        Alexander Shraer
      26. ZOOKEEPER-1355-ver6.patch
        32 kB
        Alexander Shraer
      27. ZOOKEEPER-1355-ver5.patch
        27 kB
        Alexander Shraer
      28. ZOOKEEPER-1355-ver4.patch
        27 kB
        Alexander Shraer
      29. loadbalancing-more-details.pdf
        301 kB
        Alexander Shraer
      30. ZOOKEEPER=1355-ver3.patch
        24 kB
        Alexander Shraer
      31. ZOOKEEPER-1355-ver2.patch
        22 kB
        Alexander Shraer
      32. ZOOOKEEPER-1355-test.patch
        24 kB
        Alexander Shraer
      33. ZOOOKEEPER-1355-ver1.patch
        22 kB
        Alexander Shraer
      34. ZOOOKEEPER-1355.patch
        22 kB
        Alexander Shraer
      35. loadbalancing.pdf
        288 kB
        Alexander Shraer

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Alexander Shraer
              Reporter:
              Alexander Shraer
            • Votes:
              3 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development