Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-1194

Two possible race conditions during leader establishment

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.4.0, 3.5.0
    • Component/s: server
    • Labels:
      None

      Description

      Leader.getEpochToPropose() and Leader.waitForNewEpoch() act as barriers - they make sure that a leader/follower can return from calling the method only once connectingFollowers (or electingFollowers) contain a quorum. But these methods don't make sure that the leader itself is in connectingFollowers/electingFollowers. So the leader didn't necessarily reach the barrier when followers pass it. This can cause the following problems:

      1. If the leader is not in connectingFollowers when a LearnerHandler returns from getEpochToPropose(), then the epoch sent by the leader to the follower might be smaller than the leader's own last accepted epoch.

      2. If the leader is not in electingFollowers when LearnerHandler returns from waitForNewEpoch() then the leader will send a NEWLEADER message to followers, and the followers will respond, but it is possible that the NEWLEADER message is not in outstandingProposals when these NEWLEADER acks arrive, which will cause the NEWLEADER acks to be dropped.

      To fix this I propose to explicitly check that the leader is in connectingFollowers/electingFollowers before anyone can pass these barriers.

        Attachments

        1. zookeeper-1194.patch
          11 kB
          Alexander Shraer
        2. zookeeper-1194-ver1.patch
          11 kB
          Alexander Shraer
        3. zookeeper-1194-ver2.patch
          7 kB
          Alexander Shraer

          Issue Links

            Activity

              People

              • Assignee:
                shralex Alexander Shraer
                Reporter:
                shralex Alexander Shraer
              • Votes:
                0 Vote for this issue
                Watchers:
                0 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: