[ZOOKEEPER-1144] ZooKeeperServer not starting on leader due to a race condition

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Assignee: Vishal Kher
• Reporter: Vishal Kher
• Created: 03/Aug/11 22:35
• Updated: 23/Nov/11 19:22
• Resolved: 11/Aug/11 18:10

Description

I have found one problem that is causing QuorumPeerMainTest:testQuorum to fail. This test uses 2 ZK servers.

The test is failing because leader is not starting ZooKeeperServer after leader election. so everything halts.

With the new changes, the server is now started in Leader.processAck() which is called from LeaderHandler. processAck() starts ZooKeeperServer if majority have acked NEWLEADER. The leader puts its ack in the the ackSet in Leader.lead(). Since processAck() is called from LearnerHandler it can happen that the learner's ack is processed before the leader is able to put its ack in the ackSet. When LearnerHandler invokes processAck(), the ackSet for newLeaderProposal will not have quorum (in this case 2). As a result, the ZooKeeperServer is never started on the Leader.

The leader needs to ensure that its ack is put in ackSet before starting LearnerCnxAcceptor or invoke processAck() itself after adding to ackSet. I haven't had time to go through the ZAB2 changes so I am not too familiar with the code. Can Ben/Flavio fix this?

Issue Links

• is related to: ZOOKEEPER-1125 Intermittent java core test failures
• is related to: ZOOKEEPER-1145 ObserverTest.testObserver fails at particular point after several runs of ant junt.run -Dtestcase=ObserverTest