Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-4040

java.io.IOException: Leaders epoch, 1 is less than accepted epoch, 2

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.5.5, 3.5.8, 3.6.2
    • None
    • server
    • None

    Description

      Overview (mechanically translated from ZOOKEEPER-4039):

      The acceptedEpoch is too large and the corresponding node cannot join the cluster

      After the leader receives the acceptedEpoch of more than half of the nodes, it will set its acceptedEpoch to the maximum value of these nodes plus 1, but at this time, the leader’s downtime will cause the leader node’s acceptedEpoch to be 1 larger than other nodes, and then this node will restart again Be elected as the leader, go down again, and then the remaining nodes re-elect a leader. The epoch of this leader will be smaller than the acceptedEpoch of the original leader, which causes the original node to always look and switch the follower state

      Steps to reproduce:

      3 nodes, server1, server2, server3

      Start server1, server2, and then stop server1 and server2 at the red dot below. At this time, the corresponding acceptedEpoch=1 of server2

      Restart server1, server2, and then stop server1 and server2 at the red dot below. At this time, the corresponding acceptedEpoch=2 of server2

      Restart server1, server3, wait for server1 and server3 to elect the corresponding leader as server3, and then start server2, the following exception will be repeated

      errorlog:

      java.io.IOException: Leaders epoch, 1 is less than accepted epoch, 2java.io.IOException: Leaders epoch, 1 is less than accepted epoch, 2 at org.apache.zookeeper.server.quorum.Learner.registerWithLeader(Learner.java:353) at org.apache.zookeeper.server.quorum.Follower.followLeader(Follower.java:78) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1271)2020-12-28 18:09:25,176 [myid:2] - INFO  [QuorumPeer[myid=2](plain=/0:0:0:0:0:0:0:0:2182)(secure=disabled):Follower@201] - shutdown calledjava.lang.Exception: shutdown Follower at org.apache.zookeeper.server.quorum.Follower.shutdown(Follower.java:201) at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1275)

       

      sample:

      cluster all servers server1,server2,server3

      • start server1 and server2 ,then shutdown them when they arrive below, now the accpetedEpoch of server2 is 1 , server1 is 0, server3 is 0
      • then repeat step 1 , now the accpetedEpoch of server1 is 0,server2 is 2,server3 is 0
      • then start server1 and server3 , wait unti the leader of the cluster is server3 , start server2 ,now generate the error below

      Attachments

        1. image-2020-12-28-18-20-07-842.png
          74 kB
          pengfei
        2. image-2020-12-28-18-23-14-073.png
          89 kB
          pengfei
        3. image-2020-12-28-18-25-31-960.png
          89 kB
          pengfei
        4. image-2020-12-28-18-28-07-015.png
          63 kB
          pengfei

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            pf pengfei

            Dates

              Created:
              Updated:

              Slack

                Issue deployment