Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-2478

Can not use the same Zookeeper by multiple NiFi clusters

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.0.0
    • 1.0.0, 1.0.0-Beta
    • Core Framework
    • None

    Description

      I am trying to use the same external Zookeeper from two NiFi clusters. It's expected this deployment is supported since there is 'nifi.zookeeper.root.node' setting.

      Set 'nifi.zookeeper.root.node' to '/nifi-cluster-1' for cluster-1, and '/nifi-cluster-2' for cluster-2.

      Cluster-1 started successfully. But Cluster-2 can't start with following error:

      2016-08-04 07:45:58,787 INFO [main] o.a.c.f.imps.CuratorFrameworkImpl Starting
      2016-08-04 07:45:58,794 INFO [main-EventThread] o.a.c.f.state.ConnectionStateManager State change: CONNECTED
      2016-08-04 07:45:58,795 INFO [main] o.a.n.c.c.node.CuratorNodeProtocolSender No node has yet been elected Cluster Coordinator. Cannot establish connection to cluster yet.
      2016-08-04 07:45:58,795 INFO [Curator-Framework-0] o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
      2016-08-04 07:45:58,797 WARN [main] o.a.nifi.controller.StandardFlowService Failed to connect to cluster due to: org.apache.nifi.cluster.protocol.ProtocolException: No node has yet been elected Cluster Coordinator. Cannot establish connection to cluster yet.
      org.apache.nifi.cluster.protocol.ProtocolException: No node has yet been elected Cluster Coordinator. Cannot establish connection to cluster yet.
              at org.apache.nifi.cluster.coordination.node.CuratorNodeProtocolSender.getServiceAddress(CuratorNodeProtocolSender.java:108) ~[nifi-framework-cluster-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
      

      I looked at the Zookeeper znodes and found 'leaders' is not under configured root node:

      [zk: localhost:2181(CONNECTED) 17] ls /
      [leaders, nifi-cluster-1, zookeeper]
      

      Then, I looked at the source code, and found CuratorLeaderElectionManager has following code:

              final String rootPath = zkConfig.getRootPath();
              final String leaderPath = (rootPath.endsWith("/") ? "" : "/") + "leaders/" + roleName;
      

      It doesn't include rootPath while it should.

      Attachments

        Issue Links

          Activity

            People

              markap14 Mark Payne
              ijokarumawak Koji Kawamura
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: