ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1413

Use on-disk transaction log for learner sync up

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 3.4.3
    • Fix Version/s: 3.5.0
    • Component/s: server

      Description

      Motivation:
      The learner syncs up with leader by retrieving committed log from the leader. Currently, the leader only keeps 500 entries of recently committed log in memory. If the learner falls behind more than 500 updates, the leader will send the entire snapshot to the learner.

      With the size of the snapshot for some of our Zookeeper deployments (~10G), it is prohibitively expensive to send the entire snapshot over network. Additionally, our Zookeeper may serve more than 4K updates per seconds. As a result, a network hiccups for less than a second will cause the learner to use snapshot transfer.

      Design:
      Instead of looking only at committed log in memory, the leader will also look at transaction log on disk. The amount of transaction log kept on disk is configurable and the current default is 100k. This will allow Zookeeper to tolerate longer temporal network failure before initiating the snapshot transfer.

      Implementation:
      We plan to add interface to the persistence layer will can be use to retrieve proposals from on-disk transaction log. These proposals can then be used to send to the learner using existing protocol.

      1. ZOOKEEPER-1413-3.4.patch
        61 kB
        Germán Blanco
      2. ZOOKEEPER-1413-3.4.patch
        88 kB
        Germán Blanco
      3. ZOOKEEPER-1413.patch
        23 kB
        Thawan Kooburat
      4. ZOOKEEPER-1413.patch
        70 kB
        Thawan Kooburat
      5. ZOOKEEPER-1413.patch
        87 kB
        Thawan Kooburat
      6. ZOOKEEPER-1413.patch
        87 kB
        Thawan Kooburat
      7. ZOOKEEPER-1413.patch
        87 kB
        Thawan Kooburat
      8. ZOOKEEPER-1413.patch
        87 kB
        Thawan Kooburat
      9. ZOOKEEPER-1413.patch
        88 kB
        Thawan Kooburat

        Issue Links

          Activity

          Germán Blanco made changes -
          Priority Major [ 3 ] Minor [ 4 ]
          Germán Blanco made changes -
          Issue Type Bug [ 1 ] Improvement [ 4 ]
          Germán Blanco made changes -
          Attachment ZOOKEEPER-1413-3.4.patch [ 12608247 ]
          Germán Blanco made changes -
          Fix Version/s 3.4.6 [ 12323310 ]
          Germán Blanco made changes -
          Attachment ZOOKEEPER-1413-3.4.patch [ 12608244 ]
          Germán Blanco made changes -
          Labels performance performance quorum
          Germán Blanco made changes -
          Priority Minor [ 4 ] Major [ 3 ]
          Germán Blanco made changes -
          Issue Type Improvement [ 4 ] Bug [ 1 ]
          Germán Blanco made changes -
          Fix Version/s 3.4.6 [ 12323310 ]
          Germán Blanco made changes -
          Link This issue is required by ZOOKEEPER-1777 [ ZOOKEEPER-1777 ]
          Thawan Kooburat made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Thawan Kooburat made changes -
          Attachment ZOOKEEPER-1413.patch [ 12589992 ]
          Thawan Kooburat made changes -
          Attachment ZOOKEEPER-1413.patch [ 12589375 ]
          Thawan Kooburat made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Thawan Kooburat made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Thawan Kooburat made changes -
          Attachment ZOOKEEPER-1413.patch [ 12589158 ]
          Thawan Kooburat made changes -
          Attachment ZOOKEEPER-1413.patch [ 12588995 ]
          Flavio Junqueira made changes -
          Link This issue contains ZOOKEEPER-876 [ ZOOKEEPER-876 ]
          Thawan Kooburat made changes -
          Attachment ZOOKEEPER-1413.patch [ 12583719 ]
          Thawan Kooburat made changes -
          Link This issue is depended upon by ZOOKEEPER-1710 [ ZOOKEEPER-1710 ]
          Thawan Kooburat made changes -
          Link This issue is depended upon by ZOOKEEPER-1709 [ ZOOKEEPER-1709 ]
          Thawan Kooburat made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Thawan Kooburat made changes -
          Attachment ZOOKEEPER-1413.patch [ 12583715 ]
          Patrick Hunt made changes -
          Fix Version/s 3.4.3 [ 12319288 ]
          Patrick Hunt made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Fix Version/s 3.5.0 [ 12316644 ]
          Patrick Hunt made changes -
          Assignee Thawan Kooburat [ thawan ]
          Thawan Kooburat made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Fix Version/s 3.4.3 [ 12319288 ]
          Thawan Kooburat made changes -
          Field Original Value New Value
          Attachment ZOOKEEPER-1413.patch [ 12519268 ]
          Thawan Kooburat created issue -

            People

            • Assignee:
              Thawan Kooburat
              Reporter:
              Thawan Kooburat
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development