Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2467

Zookeeper Puppet script does not setup Zookeeper node ids correctly

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.1.0
    • None
    • deployment
    • None
    • Ubuntu Linux (though likely other environments, as well)

    Description

      In order to increase Zookeeper quorum, one needs to add a list of nodes to the zoo.cfg in the following format:

      server.<node id>=<ip address>:2888:3888
      server.<node id>=<ip address>:2888:3888
      ...

      A Zookeeper node assumes that the first entry in the list is itself, and it assumes that the <node id> it has for a node is consistent across nodes. A working setup for a node with id "2" might look like this:

      server.2=10.0.1.24:2888:3888
      server.0=10.0.1.35:2888:3888
      server.1=10.0.1.12:2888:3888

      Bigtop's template for zoo.cfg does the following, however:

      <% @ensemble.each_with_index do |server,idx| %>
      server.<%= idx %>=<%= server %>
      <% end %>

      "ensemble" is simply an array of strings in the format "<ip address>:2888:3888". Note that the template uses ruby to auto generate an index for each entry, so the first entry in the list will always have an id of "0".

      This means that, when the nodes try to talk to each other, they all claim to be peer 0, and they fail to connect with the following error:

      2016-06-02 18:26:01,195 [myid:0] - INFO [WorkerSender[myid=0]:QuorumCnxManager@193] - Have smaller server identifier, so dropping the connection: (1, 0)

      I'm working on a proposed patch for this issue, where the ensemble variable takes an array of two element arrays; each sub array would be an id and a properly formatted ip string. I'm definitely open to other ideas, though.

      Attachments

        Issue Links

          Activity

            People

              petevg Pete Vander Giessen
              petevg Pete Vander Giessen
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: