ZooKeeper
  1. ZooKeeper
  2. ZOOKEEPER-1378

Provide option to turn off sending of diffs

    Details

    • Type: Task Task
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.5.1
    • Component/s: None
    • Labels:
      None

      Description

      From Patrick:
      we need to have an option to turn off sending of diffs. There are a couple of really strong reasons I can think of to do this:

      1) 3.3.x is broken in a similar way, there is an upgrade problem we can't solve short of having ppl first upgrade to a fixed 3.3 (3.3.5 say) and then upgrading to 3.4.x. If we could turn off diff sending this would address the problem.

      2) safety valve. Say we find another new problem with diff sending in 3.4/3/5. Having an option to turn it off would be useful for people as a workaround until a fix is found and released.

        Activity

        Hide
        Benjamin Reed added a comment -

        i'm not sure if this is such a good idea. if people start enabling this option "just to be sure", we are going to run into a dual problem of 1) less production experience for the DIFF code path 2) other performance problems with always doing a diff. the second problem can be particularly bad since diffs make the initial sync with the leader very fast. if you always cause a state transfer (aka snapshot) to happen, you will increase the network load, leader load, and sync time. and these increases will grow linearly with the size of your database. i'm a -0 on this one.

        Show
        Benjamin Reed added a comment - i'm not sure if this is such a good idea. if people start enabling this option "just to be sure", we are going to run into a dual problem of 1) less production experience for the DIFF code path 2) other performance problems with always doing a diff. the second problem can be particularly bad since diffs make the initial sync with the leader very fast. if you always cause a state transfer (aka snapshot) to happen, you will increase the network load, leader load, and sync time. and these increases will grow linearly with the size of your database. i'm a -0 on this one.
        Hide
        Patrick Hunt added a comment -

        Ben what you mention is indeed a concern, however it can be mitigated by removing the option once we move to 3.5.

        3.3 has this same flaw (ZOOKEEPER-1367). My hope is to not require everyone to upgrade from pre 3.3.5 to 3.3.5, then to 3.4.x. Do you have another solution/option?

        Show
        Patrick Hunt added a comment - Ben what you mention is indeed a concern, however it can be mitigated by removing the option once we move to 3.5. 3.3 has this same flaw ( ZOOKEEPER-1367 ). My hope is to not require everyone to upgrade from pre 3.3.5 to 3.3.5, then to 3.4.x. Do you have another solution/option?
        Hide
        Mahadev konar added a comment -

        Pat/Ben,
        Can we mark this as wont fix?

        Show
        Mahadev konar added a comment - Pat/Ben, Can we mark this as wont fix?
        Hide
        Mahadev konar added a comment -

        Moving it out to 3.5. I think we should mark it as wont fix but Ill keep it open for now.

        Show
        Mahadev konar added a comment - Moving it out to 3.5. I think we should mark it as wont fix but Ill keep it open for now.
        Hide
        Michi Mutsuzaki added a comment -

        It looks like this issue is no longer relevant. Can we close this?

        Show
        Michi Mutsuzaki added a comment - It looks like this issue is no longer relevant. Can we close this?

          People

          • Assignee:
            Unassigned
            Reporter:
            Ted Yu
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development