[STORM-466] Storm build failures due to metrics_test.clj

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Duplicate
• Component/s: storm-core
• Assignee: Unassigned
• Reporter: Harsha
• Created: 23/Aug/14 20:58
• Updated: 05/Oct/15 01:55
• Resolved: 25/Aug/14 15:27

Description

As reported by Rick Kilgore on mailing list
I need to make a couple modifications to KafkaSpout for our needs and I’m planning to submit back a PR to the main repo. But I can’t build. I’ve installed the latest git, java 7, and mvn 3 via apt-get onto Ubuntu 14.04 LTS. Then I run:

git clone 'https://github.com/apache/incubator-storm' storm
cd storm
mvn clean install

I get lots of exceptions, but I think most are expected as part of the tests. Below is the output at the end that I’m guessing is not good. I’ve also tried this on a Mac running Mavericks and on an Amazon Linux installation. I’ve tried in my usual shell setup, and with a stripped down bash setup with a minimal PATH. The results are mostly the same, but regardless always fails in some way. This is with what’s in master now, without any changes by me. Here is the tail end of my output from mvn install:

187855 [Thread-287] INFO backtype.storm.daemon.worker - Worker 6b378fbc-cbcd-4ba4-aed8-9d8ad247eb2f for storm shell-metrics-tester-1-3 on 2e862cd2-ccaf-48ac-820e-6c5a714e9bc8:1024 has finished loading
187857 [Thread-291-2] INFO backtype.storm.task.ShellBolt - Launched subprocess with pid 16985
187858 [Thread-291-2] INFO backtype.storm.daemon.executor - Prepared bolt 2:(2)
187873 [Thread-302] INFO backtype.storm.task.ShellBolt - ShellLog pid:16985, name:2 bolt initializing
188651 [main] INFO backtype.storm.daemon.nimbus - Shutting down master
188653 [main] INFO backtype.storm.daemon.nimbus - Shut down master
188653 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN org.apache.zookeeper.server.NIOServerCnxn - caught end of stream exception
org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x14801d52e3d0001, likely client has
closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [zookeeper-3.4.6.jar:3.4.6-1569965]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
188658 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN org.apache.zookeeper.server.NIOServerCnxn - caught end of stream exception
org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x14801d52e3d0003, likely client has
closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [zookeeper-3.4.6.jar:3.4.6-1569965]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
188660 [main] INFO backtype.storm.daemon.supervisor - Shutting down 2e862cd2-ccaf-48ac-820e-6c5a714e9bc8:6b378fbc-cbcd-4ba4-aed8-9d8ad247eb2f
188660 [main] INFO backtype.storm.process-simulator - Killing process d2a10e11-adbd-49cd-889c-5137777fdc56
188660 [main] INFO backtype.storm.daemon.worker - Shutting down worker shell-metrics-tester-1-3 2e862cd2-ccaf-48ac-820e-6c5a714e9bc8 1024
188660 [main] INFO backtype.storm.daemon.worker - Shutting down receive thread
188660 [main] INFO backtype.storm.messaging.loader - Shutting down receiving-thread: [shell-metrics-tester-1-3, 1024]
188660 [main] INFO backtype.storm.messaging.loader - Waiting for receiving-thread:[shell-metrics-tester-1-3, 1024] to die
188660 [main] INFO backtype.storm.messaging.loader - Shutdown receiving-thread: [shell-metrics-tester-1-3, 1024]
188660 [main] INFO backtype.storm.daemon.worker - Shut down receive thread
188660 [main] INFO backtype.storm.daemon.worker - Terminating messaging context
188660 [main] INFO backtype.storm.daemon.worker - Shutting down executors
188660 [main] INFO backtype.storm.daemon.executor - Shutting down executor 2:[2 2]
188661 [Thread-300-worker-receiver-thread-0] INFO backtype.storm.messaging.loader - Receiving-thread:[shell-metrics-tester-1-3, 1024] received shutdown notice
188661 [Thread-291-2] INFO backtype.storm.util - Async loop interrupted!
188661 [Thread-290-disruptor-executor[2 2]-send-queue] INFO backtype.storm.util - Async loop interrupted!
188662 [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2000] WARN org.apache.zookeeper.server.NIOServerCnxn - caught end of stream exception
org.apache.zookeeper.server.ServerCnxn$EndOfStreamException: Unable to read additional data from client sessionid 0x14801d52e3d0005, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228) ~[zookeeper-3.4.6.jar:3.4.6-1569965]
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208) [zookeeper-3.4.6.jar:3.4.6-1569965]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
188663 [main] INFO backtype.storm.daemon.executor - Shut down executor 2:[2 2]
188663 [main] INFO backtype.storm.daemon.executor - Shutting down executor __acker:[3 3]
188673 [Thread-293-__acker] INFO backtype.storm.util - Async loop interrupted!
188674 [Thread-302] ERROR backtype.storm.task.ShellBolt - Halting process: ShellBolt died.
java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
(Unable to capture error stream)

at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:101) ~[classes/:na]
at backtype.storm.task.ShellBolt$1.run(ShellBolt.java:116) ~[classes/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
188674 [Thread-302] ERROR backtype.storm.daemon.executor -
java.lang.RuntimeException: backtype.storm.multilang.NoOutputException: Pipe to subprocess seems to be broken! No output read.
Serializer Exception:
(Unable to capture error stream)

at backtype.storm.utils.ShellProcess.readShellMsg(ShellProcess.java:101) ~[classes/:na]
at backtype.storm.task.ShellBolt$1.run(ShellBolt.java:116) ~[classes/:na]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_65]
188677 [Thread-292-disruptor-executor[3 3]-send-queue] INFO backtype.storm.util - Async loop interrupted!
188677 [main] INFO backtype.storm.daemon.executor - Shut down executor __acker:[3 3]
188678 [main] INFO backtype.storm.daemon.executor - Shutting down executor __metricsclojure.storm.metric.testing.FakeMetricConsumer:[4 4]

Issue Links

• duplicates: STORM-449 ShellBolt can cause periodic test failures