Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20404

ZooKeeper quorum fails to start due to missing log4j library

    XMLWordPrintableJSON

    Details

    • Release Note:
      Hide
      The Zookeeper scripts in the Flink distribution have been modified to disable the Log4j JMX integration due to an incompatibility between Zookeeper 3.4 and Log4j 2.
      To re-enable this feature, remove the line in the "zookeeper.sh" file that sets "zookeeper.jmx.log4j.disable".
      Show
      The Zookeeper scripts in the Flink distribution have been modified to disable the Log4j JMX integration due to an incompatibility between Zookeeper 3.4 and Log4j 2. To re-enable this feature, remove the line in the "zookeeper.sh" file that sets "zookeeper.jmx.log4j.disable".

      Description

      Upon starting a zookeeper quorum using flink's bootstrapped zookeeper, it throws the following exception.

       

      2020-11-27 13:13:38,371 ERROR org.apache.flink.runtime.zookeeper.FlinkZooKeeperQuorumPeer  [] - Error running ZooKeeper quorum peer: org/apache/log4j/jmx/HierarchyDynamicMBean
      java.lang.NoClassDefFoundError: org/apache/log4j/jmx/HierarchyDynamicMBean
              at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.jmx.ManagedUtil.registerLog4jMBeans(ManagedUtil.java:51) ~[flink-shaded-zookeeper-3.4.14.jar:3.4.14-11.0]
              at org.apache.flink.shaded.zookeeper3.org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:125) ~[flink-shaded-zookeeper-3.4.14.jar:3.4.14-11.0]
              at org.apache.flink.runtime.zookeeper.FlinkZooKeeperQuorumPeer.runFlinkZkQuorumPeer(FlinkZooKeeperQuorumPeer.java:123) ~[flink-dist_2.11-1.11.2.jar:1.11.2]
              at org.apache.flink.runtime.zookeeper.FlinkZooKeeperQuorumPeer.main(FlinkZooKeeperQuorumPeer.java:79) [flink-dist_2.11-1.11.2.jar:1.11.2]
      Caused by: java.lang.ClassNotFoundException: org.apache.log4j.jmx.HierarchyDynamicMBean
              at java.net.URLClassLoader.findClass(URLClassLoader.java:382) ~[?:1.8.0_262]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:418) ~[?:1.8.0_262]
              at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) ~[?:1.8.0_262]
              at java.lang.ClassLoader.loadClass(ClassLoader.java:351) ~[?:1.8.0_262]
              ... 4 more
      

      This happens because the new flink version is missing a log4j library. This can be solved by adding log4j-1.2.17.jar to the classpath, nonetheless the bootstrapped zookeepeer version should be compatible with the log4j2 libraries that come with flink's default installation.

       

      Steps to reproduce:

      1. Fresh install of flink version 1.11.2 
      2. Change the zookeeper config to start as a quorum
        server.1=<hostname>:2888:3888
        server.2=<hostname>:2888:3888
      1. Start zookeeper
      2. <installation_path>/bin/zookeeper.sh start-foreground 1

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                chesnay Chesnay Schepler
                Reporter:
                PedroMrChaves Pedro Miguel Rainho Chaves
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: