Uploaded image for project: 'ZooKeeper'
  1. ZooKeeper
  2. ZOOKEEPER-3830

After add a new node, zookeeper cluster won't commit any proposal if this new node is leader

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Duplicate
    • 3.6.1, 3.5.8
    • 3.5.9, 3.7.0, 3.6.2
    • server
    • None
    • Zookeeper 3.5.8

      JDK 1.8

    Description

      I have a zookeeper cluster with 3 nodes, node3 is the leader of the cluster.

       

      server.1=node1
      server.2=node2
      server.3=node3 # current leader

      With dynamic reconfiguration disabled, I scale this cluster to 4 nodes with 2 steps:

      1. Start node4 with new config, now node4 is a follower.
      2. Modify config and restart node1, node2 and node3 one by one.

      The new cluster config is:

      server.1=node1
      server.2=node2
      server.3=node3 
      server.4=node4 # current leader
      

      After restart, node4 is the leader of this cluster. But I cannot connect to this cluster using zkCli now, zkCli keeps in CONNECTING status.

      If I restart node4, node3 will be the new leader, and now I can connected to cluster using zkCli again.

      After some digging, I find node4's Leader#allowedToCommit field is false, so this cluster won't commit any new proposals.

       

      I have attached a zookeeper cluster to reproduce this problem. The cluster in the attachment can run in one single machine.

       

      Attachments

        1. reproduce-zkclusters.tar.gz
          35.96 MB
          Keli Wang

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            symat Mate Szalay-Beko
            keliwang Keli Wang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment