ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-965

Need a multi-update command to allow multiple znodes to be updated safely

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 3.3.3
    • Fix Version/s: 3.4.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The basic idea is to have a single method called "multi" that will accept a list of create, delete, update or check objects each of which has a desired version or file state in the case of create. If all of the version and existence constraints can be satisfied, then all updates will be done atomically.

      Two API styles have been suggested. One has a list as above and the other style has a "Transaction" that allows builder-like methods to build a set of updates and a commit method to finalize the transaction. This can trivially be reduced to the first kind of API so the list based API style should be considered the primitive and the builder style should be implemented as syntactic sugar.

      The total size of all the data in all updates and creates in a single transaction should be limited to 1MB.

      Implementation-wise this capability can be done using standard ZK internals. The changes include:

      • update to ZK clients to all the new call
      • additional wire level request
      • on the server, in the code that converts transactions to idempotent form, the code should be slightly extended to convert a list of operations to idempotent form.
      • on the client, a down-rev server that rejects the multi-update should be detected gracefully and an informative exception should be thrown.

      To facilitate shared development, I have established a github repository at https://github.com/tdunning/zookeeper and am happy to extend committer status to anyone who agrees to donate their code back to Apache. The final patch will be attached to this bug as normal.

      1. ZOOKEEPER-965.patch
        174 kB
        Ted Dunning
      2. ZOOKEEPER-965.patch
        174 kB
        Ted Dunning
      3. ZOOKEEPER-965.patch
        174 kB
        Ted Dunning
      4. ZOOKEEPER-965.patch
        172 kB
        Ted Dunning
      5. ZOOKEEPER-965.patch
        172 kB
        Ted Dunning
      6. ZOOKEEPER-965.patch
        253 kB
        Ted Dunning
      7. ZOOKEEPER-965.patch
        172 kB
        Ted Dunning
      8. ZOOKEEPER-965.patch
        176 kB
        Ted Dunning
      9. ZOOKEEPER-965.patch
        167 kB
        Ted Dunning
      10. ZOOKEEPER-965.patch
        169 kB
        Ted Dunning
      11. ZOOKEEPER-965.patch
        166 kB
        Ted Dunning
      12. ZOOKEEPER-965.patch
        162 kB
        Ted Dunning
      13. ZOOKEEPER-965.patch
        151 kB
        Ted Dunning
      14. ZOOKEEPER-965.patch
        154 kB
        Ted Dunning
      15. ZOOKEEPER-965.patch
        151 kB
        Ted Dunning
      16. ZOOKEEPER-965.patch
        133 kB
        Ted Dunning
      17. ZOOKEEPER-965.patch
        135 kB
        Ted Dunning
      18. ZOOKEEPER-965.patch
        133 kB
        Ted Dunning
      19. ZOOKEEPER-965.patch
        128 kB
        Ted Dunning
      20. ZOOKEEPER-965.patch
        128 kB
        Ted Dunning
      21. ZOOKEEPER-965.patch
        150 kB
        Ted Dunning

        Issue Links

        1.
        Client side for multi Sub-task Closed Unassigned
         
        2.
        Server side decoding and function dispatch Sub-task Closed Unassigned
         
        3.
        Database multi-update Sub-task Closed Unassigned
         

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Ted Dunning
              Reporter:
              Ted Dunning
            • Votes:
              0 Vote for this issue
              Watchers:
              14 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development