Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-146

Unit test regression when storm is compiled with 3.4.5 zookeeper

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 0.9.2-incubating
    • storm-core
    • None

    Description

      https://github.com/nathanmarz/storm/issues/480

      commit d0473a06a32815e1945b2fffe5af395fb9432192
      Author: Jason Jackson <jackson@twitter.com>
      Date: Fri Feb 1 00:49:01 2013 -0800

      upgrade Zookeeper 3.4.5

      diff --git a/project.clj b/project.clj
      index 7f59387..725f8c9 100644
      — a/project.clj
      +++ b/project.clj
      @@ -9,7 +9,9 @@
      :exclusions [org.slf4j/slf4j-api]]
      [clj-time "0.4.1"]
      [com.netflix.curator/curator-framework "1.0.1"

      • :exclusions [log4j/log4j]]
        + :exclusions [log4j/log4j org.apache.zookeeper/zookeeper]]
        + [org.apache.zookeeper/zookeeper "3.4.5"
        + :exclusions [com.sun.jmx/jmxri com.sun.jdmk/jmxtools javax.jms/jms log4j/log4j jline org.slf4j/slf4j-log4j12]]
        [backtype/jzmq "2.1.0"]
        [com.googlecode.json-simple/json-simple "1.1"]
        [compojure "1.1.3"]
        diff --git a/src/clj/backtype/storm/zookeeper.clj b/src/clj/backtype/storm/zookeeper.clj
        index 76858a7..c591cf6 100644
          • a/src/clj/backtype/storm/zookeeper.clj
            +++ b/src/clj/backtype/storm/zookeeper.clj
            @@ -6,7 +6,7 @@
            ZooDefs ZooDefs$Ids CreateMode WatchedEvent Watcher$Event Watcher$Event$KeeperState
            Watcher$Event$EventType KeeperException$NodeExistsException])
            (:import [org.apache.zookeeper.data Stat])
      • (:import [org.apache.zookeeper.server ZooKeeperServer NIOServerCnxn$Factory])
        + (:import [org.apache.zookeeper.server ZooKeeperServer NIOServerCnxnFactory])
        (:import [java.net InetSocketAddress BindException])
        (:import [java.io File])
        (:import [backtype.storm.utils Utils ZookeeperAuthInfo])
        @@ -132,7 +132,7 @@
        (let [localfile (File. localdir)
        zk (ZooKeeperServer. localfile localfile 2000)
        [retport factory] (loop [retport (if port port 2000)]
      • (if-let [factory-tmp (try-cause (NIOServerCnxn$Factory. (InetSocketAddress. retport))
        + (if-let [factory-tmp (try-cause (doto (NIOServerCnxnFactory.) (.configure (InetSocketAddress. retport) 0))
        (catch BindException e
        (when (> (inc retport) (if port port 65535))
        (throw (RuntimeException. "No port is available to launch an inprocess zookeeper.")))))]
        diff --git a/src/jvm/backtype/storm/utils/Utils.java b/src/jvm/backtype/storm/utils/Utils.java
        index 36d4d5c..aacdd8f 100644
          • a/src/jvm/backtype/storm/utils/Utils.java
            +++ b/src/jvm/backtype/storm/utils/Utils.java
            @@ -293,8 +293,8 @@ public class Utils { serverPorts.add(zkServer + ":" + Utils.getInt(port)); }

            String zkStr = StringUtils.join(serverPorts, ",") + root;
            +
            try {

      • CuratorFrameworkFactory.Builder builder = CuratorFrameworkFactory.builder()
        .connectString(zkStr)
        .connectionTimeoutMs(Utils.getInt(conf.get(Config.STORM_ZOOKEEPER_CONNECTION_TIMEOUT)))
        This patch causes transactional-test to hang non-deterministically.
        Also if you try to upgrade both curator and zk same thing.
        Also, zk inprocess is much slower in 3.4.x branch,
        one way to solve this is to reuse inprocess zookeeper's instead of constantly starting them up and shutting them down in unit tests.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              xumingming James Xu
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: