Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-1194

Two possible race conditions during leader establishment

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 3.4.0, 3.5.0
    • server
    • None

    Description

      Leader.getEpochToPropose() and Leader.waitForNewEpoch() act as barriers - they make sure that a leader/follower can return from calling the method only once connectingFollowers (or electingFollowers) contain a quorum. But these methods don't make sure that the leader itself is in connectingFollowers/electingFollowers. So the leader didn't necessarily reach the barrier when followers pass it. This can cause the following problems:

      1. If the leader is not in connectingFollowers when a LearnerHandler returns from getEpochToPropose(), then the epoch sent by the leader to the follower might be smaller than the leader's own last accepted epoch.

      2. If the leader is not in electingFollowers when LearnerHandler returns from waitForNewEpoch() then the leader will send a NEWLEADER message to followers, and the followers will respond, but it is possible that the NEWLEADER message is not in outstandingProposals when these NEWLEADER acks arrive, which will cause the NEWLEADER acks to be dropped.

      To fix this I propose to explicitly check that the leader is in connectingFollowers/electingFollowers before anyone can pass these barriers.

      Attachments

        1. zookeeper-1194.patch
          11 kB
          Alexander Shraer
        2. zookeeper-1194-ver1.patch
          11 kB
          Alexander Shraer
        3. zookeeper-1194-ver2.patch
          7 kB
          Alexander Shraer

        Issue Links

          Activity

            People

              shralex Alexander Shraer
              shralex Alexander Shraer
              Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: