[ZOOKEEPER-1194] Two possible race conditions during leader establishment

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.4.0, 3.5.0
• Component/s: server
• Assignee: Alexander Shraer
• Reporter: Alexander Shraer
• Created: 20/Sep/11 00:10
• Updated: 23/Nov/11 19:22
• Resolved: 05/Nov/11 06:38

Description

Leader.getEpochToPropose() and Leader.waitForNewEpoch() act as barriers - they make sure that a leader/follower can return from calling the method only once connectingFollowers (or electingFollowers) contain a quorum. But these methods don't make sure that the leader itself is in connectingFollowers/electingFollowers. So the leader didn't necessarily reach the barrier when followers pass it. This can cause the following problems:

1. If the leader is not in connectingFollowers when a LearnerHandler returns from getEpochToPropose(), then the epoch sent by the leader to the follower might be smaller than the leader's own last accepted epoch.

2. If the leader is not in electingFollowers when LearnerHandler returns from waitForNewEpoch() then the leader will send a NEWLEADER message to followers, and the followers will respond, but it is possible that the NEWLEADER message is not in outstandingProposals when these NEWLEADER acks arrive, which will cause the NEWLEADER acks to be dropped.

To fix this I propose to explicitly check that the leader is in connectingFollowers/electingFollowers before anyone can pass these barriers.

Issue Links

• is part of: ZOOKEEPER-1270 testEarlyLeaderAbandonment failing intermittently, quorum formed, no serving.