[ZOOKEEPER-2481] Flaky Test: testZeroWeightQuorum - ASF JIRA

XML

Word

Printable

JSON

Details

Type: Sub-task
Status: Closed
Priority: Major
Resolution: Cannot Reproduce
Affects Version/s: 3.4.8, 3.5.2
Fix Version/s: 3.5.5
Component/s: server, tests
Labels:
- flaky
- flaky-test

Description

See https://builds.apache.org/job/ZooKeeper-trunk-openjdk7/1098/

Error Message

Threads didn't join
Stacktrace

junit.framework.AssertionFailedError: Threads didn't join
	at org.apache.zookeeper.test.FLEZeroWeightTest.testZeroWeightQuorum(FLEZeroWeightTest.java:167)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
Standard Output

2016-07-21 04:24:14,065 [myid:] - INFO  [main:JUnit4ZKTestRunner@47] - No test.method specified. using default methods.
2016-07-21 04:24:14,158 [myid:] - INFO  [main:JUnit4ZKTestRunner@47] - No test.method specified. using default methods.
2016-07-21 04:24:14,176 [myid:] - INFO  [main:ZKTestCase$1@55] - STARTING testZeroWeightQuorum
2016-07-21 04:24:14,180 [myid:] - INFO  [main:JUnit4ZKTestRunner$LoggedInvokeMethod@77] - RUNNING TEST METHOD testZeroWeightQuorum
2016-07-21 04:24:14,180 [myid:] - INFO  [main:FLEZeroWeightTest@143] - TestZeroWeightQuorum: testZeroWeightQuorum, 9
2016-07-21 04:24:14,183 [myid:] - INFO  [main:PortAssignment@157] - Single test process using ports from 11221 - 32767.
2016-07-21 04:24:14,187 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11222 from range 11221 - 32767.
2016-07-21 04:24:14,189 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11223 from range 11221 - 32767.
2016-07-21 04:24:14,189 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11224 from range 11221 - 32767.
2016-07-21 04:24:14,218 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11225 from range 11221 - 32767.
2016-07-21 04:24:14,218 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11226 from range 11221 - 32767.
2016-07-21 04:24:14,219 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11227 from range 11221 - 32767.
2016-07-21 04:24:14,219 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11228 from range 11221 - 32767.
2016-07-21 04:24:14,220 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11229 from range 11221 - 32767.
2016-07-21 04:24:14,220 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11230 from range 11221 - 32767.
2016-07-21 04:24:14,221 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11231 from range 11221 - 32767.
2016-07-21 04:24:14,224 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11232 from range 11221 - 32767.
2016-07-21 04:24:14,224 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11233 from range 11221 - 32767.
2016-07-21 04:24:14,225 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11234 from range 11221 - 32767.
2016-07-21 04:24:14,225 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11235 from range 11221 - 32767.
2016-07-21 04:24:14,225 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11236 from range 11221 - 32767.
2016-07-21 04:24:14,226 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11237 from range 11221 - 32767.
2016-07-21 04:24:14,226 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11238 from range 11221 - 32767.
2016-07-21 04:24:14,227 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11239 from range 11221 - 32767.
2016-07-21 04:24:14,227 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11240 from range 11221 - 32767.
2016-07-21 04:24:14,228 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11241 from range 11221 - 32767.
2016-07-21 04:24:14,228 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11242 from range 11221 - 32767.
2016-07-21 04:24:14,229 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11243 from range 11221 - 32767.
2016-07-21 04:24:14,229 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11244 from range 11221 - 32767.
2016-07-21 04:24:14,229 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11245 from range 11221 - 32767.
2016-07-21 04:24:14,230 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11246 from range 11221 - 32767.
2016-07-21 04:24:14,230 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11247 from range 11221 - 32767.
2016-07-21 04:24:14,231 [myid:] - INFO  [main:PortAssignment@85] - Assigned port 11248 from range 11221 - 32767.
2016-07-21 04:24:14,235 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,262 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,273 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11224
2016-07-21 04:24:14,309 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,315 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,328 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11223
2016-07-21 04:24:14,329 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-0
2016-07-21 04:24:14,329 [myid:] - INFO  [Thread-0:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,330 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,331 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,331 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11227
2016-07-21 04:24:14,332 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,351 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,353 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-2
2016-07-21 04:24:14,353 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11226
2016-07-21 04:24:14,354 [myid:] - INFO  [Thread-2:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,355 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,355 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,356 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11230
2016-07-21 04:24:14,357 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,359 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,368 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-3
2016-07-21 04:24:14,368 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11229
2016-07-21 04:24:14,369 [myid:] - INFO  [Thread-3:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,370 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,370 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,371 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11233
2016-07-21 04:24:14,372 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,373 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,376 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-4
2016-07-21 04:24:14,376 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11232
2016-07-21 04:24:14,376 [myid:] - INFO  [Thread-4:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,377 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,378 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,378 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11236
2016-07-21 04:24:14,379 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,381 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,383 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-5
2016-07-21 04:24:14,383 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11235
2016-07-21 04:24:14,383 [myid:] - INFO  [Thread-5:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,384 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,385 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,385 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11239
2016-07-21 04:24:14,386 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,388 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,390 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-6
2016-07-21 04:24:14,391 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11238
2016-07-21 04:24:14,391 [myid:] - INFO  [Thread-6:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,392 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,392 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,392 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11242
2016-07-21 04:24:14,393 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,395 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,397 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-7
2016-07-21 04:24:14,398 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11241
2016-07-21 04:24:14,398 [myid:] - INFO  [Thread-7:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,399 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,399 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,400 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11245
2016-07-21 04:24:14,400 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,402 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,404 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11244
2016-07-21 04:24:14,405 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-8
2016-07-21 04:24:14,405 [myid:] - INFO  [Thread-8:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,406 [myid:] - INFO  [main:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,407 [myid:] - INFO  [main:NIOServerCnxnFactory@673] - Configuring NIO connection handler with 10s sessionless connection timeout, 1 selector thread(s), 8 worker threads, and 64 kB direct buffers.
2016-07-21 04:24:14,407 [myid:] - INFO  [main:NIOServerCnxnFactory@686] - binding to port /127.0.0.1:11248
2016-07-21 04:24:14,408 [myid:] - INFO  [main:QuorumPeer@776] - currentEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,410 [myid:] - INFO  [main:QuorumPeer@791] - acceptedEpoch not found! Creating with a reasonable default of 0. This should only happen when you are upgrading your installation
2016-07-21 04:24:14,412 [myid:] - INFO  [QuorumPeerListener:QuorumCnxManager$Listener@632] - My election bind port: /127.0.0.1:11247
2016-07-21 04:24:14,412 [myid:] - INFO  [main:FLEZeroWeightTest$LEThread@101] - Constructor: Thread-9
2016-07-21 04:24:14,412 [myid:] - INFO  [main:FLEZeroWeightTest@162] - Started threads testZeroWeightQuorum
2016-07-21 04:24:14,412 [myid:] - INFO  [Thread-9:FLEZeroWeightTest$LEThread@112] - Going to call leader election.
2016-07-21 04:24:14,431 [myid:] - INFO  [Thread-0:FastLeaderElection@894] - New election. My id =  0, proposed zxid=0x0
2016-07-21 04:24:14,431 [myid:] - WARN  [Thread-2:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,431 [myid:] - WARN  [Thread-7:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,432 [myid:] - WARN  [Thread-3:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,432 [myid:] - WARN  [Thread-2:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,434 [myid:] - WARN  [Thread-9:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,436 [myid:] - WARN  [Thread-9:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,434 [myid:] - WARN  [Thread-8:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,437 [myid:] - WARN  [Thread-8:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,433 [myid:] - WARN  [Thread-6:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,437 [myid:] - WARN  [Thread-6:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,433 [myid:] - WARN  [Thread-7:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,433 [myid:] - WARN  [Thread-3:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,433 [myid:] - WARN  [Thread-5:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,439 [myid:] - WARN  [Thread-5:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,432 [myid:] - WARN  [Thread-4:MBeanRegistry@112] - Failed to register MBean LeaderElection
2016-07-21 04:24:14,439 [myid:] - WARN  [Thread-4:FastLeaderElection@876] - Failed to register with JMX
javax.management.InstanceAlreadyExistsException: org.apache.ZooKeeperService:name0=LeaderElection
	at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:437)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerWithRepository(DefaultMBeanServerInterceptor.java:1898)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:966)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:900)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:324)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:522)
	at org.apache.zookeeper.jmx.MBeanRegistry.register(MBeanRegistry.java:108)
	at org.apache.zookeeper.server.quorum.FastLeaderElection.lookForLeader(FastLeaderElection.java:873)
	at org.apache.zookeeper.test.FLEZeroWeightTest$LEThread.run(FLEZeroWeightTest.java:113)
2016-07-21 04:24:14,439 [myid:] - INFO  [Thread-5:FastLeaderElection@894] - New election. My id =  4, proposed zxid=0x0
2016-07-21 04:24:14,439 [myid:] - INFO  [Thread-3:FastLeaderElection@894] - New election. My id =  2, proposed zxid=0x0
2016-07-21 04:24:14,438 [myid:] - INFO  [Thread-7:FastLeaderElection@894] - New election. My id =  6, proposed zxid=0x0
2016-07-21 04:24:14,438 [myid:] - INFO  [WorkerReceiver[myid=0]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,438 [myid:] - INFO  [Thread-6:FastLeaderElection@894] - New election. My id =  5, proposed zxid=0x0
2016-07-21 04:24:14,440 [myid:] - INFO  [WorkerReceiver[myid=0]:FastLeaderElection@688] - Notification: 2 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,437 [myid:] - INFO  [Thread-8:FastLeaderElection@894] - New election. My id =  7, proposed zxid=0x0
2016-07-21 04:24:14,437 [myid:] - INFO  [Thread-9:FastLeaderElection@894] - New election. My id =  8, proposed zxid=0x0
2016-07-21 04:24:14,436 [myid:] - INFO  [Thread-2:FastLeaderElection@894] - New election. My id =  1, proposed zxid=0x0
2016-07-21 04:24:14,439 [myid:] - INFO  [Thread-4:FastLeaderElection@894] - New election. My id =  3, proposed zxid=0x0
2016-07-21 04:24:14,446 [myid:] - INFO  [/127.0.0.1:11226:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:51059
2016-07-21 04:24:14,446 [myid:] - INFO  [/127.0.0.1:11223:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:42458
2016-07-21 04:24:14,447 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (1, 0)
2016-07-21 04:24:14,449 [myid:] - INFO  [/127.0.0.1:11229:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:60297
2016-07-21 04:24:14,449 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (2, 0)
2016-07-21 04:24:14,450 [myid:] - INFO  [/127.0.0.1:11232:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:45441
2016-07-21 04:24:14,450 [myid:] - WARN  [SendWorker:0:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,452 [myid:] - INFO  [/127.0.0.1:11226:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:51068
2016-07-21 04:24:14,451 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (3, 0)
2016-07-21 04:24:14,452 [myid:] - WARN  [SendWorker:0:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 0 my id = 1
2016-07-21 04:24:14,453 [myid:] - INFO  [WorkerReceiver[myid=1]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,454 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (2, 1)
2016-07-21 04:24:14,455 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (3, 1)
2016-07-21 04:24:14,456 [myid:] - INFO  [/127.0.0.1:11235:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:59529
2016-07-21 04:24:14,456 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (4, 0)
2016-07-21 04:24:14,457 [myid:] - INFO  [/127.0.0.1:11238:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:43251
2016-07-21 04:24:14,457 [myid:] - WARN  [RecvWorker:0:QuorumCnxManager$RecvWorker@915] - Connection broken for id 0, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,457 [myid:] - WARN  [RecvWorker:0:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,457 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (5, 0)
2016-07-21 04:24:14,458 [myid:] - INFO  [/127.0.0.1:11241:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:44371
2016-07-21 04:24:14,458 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (6, 0)
2016-07-21 04:24:14,459 [myid:] - INFO  [/127.0.0.1:11244:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:40443
2016-07-21 04:24:14,459 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (7, 0)
2016-07-21 04:24:14,460 [myid:] - INFO  [/127.0.0.1:11247:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:57714
2016-07-21 04:24:14,460 [myid:] - INFO  [WorkerSender[myid=0]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (8, 0)
2016-07-21 04:24:14,460 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (5, 1)
2016-07-21 04:24:14,461 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@688] - Notification: 2 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,461 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (6, 1)
2016-07-21 04:24:14,461 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (7, 1)
2016-07-21 04:24:14,462 [myid:] - INFO  [WorkerSender[myid=1]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (8, 1)
2016-07-21 04:24:14,466 [myid:] - WARN  [RecvWorker:0:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,469 [myid:] - INFO  [/127.0.0.1:11226:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:51082
2016-07-21 04:24:14,475 [myid:] - WARN  [SendWorker:0:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,476 [myid:] - WARN  [SendWorker:0:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 0 my id = 4
2016-07-21 04:24:14,479 [myid:] - INFO  [WorkerReceiver[myid=1]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,479 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@688] - Notification: 2 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,482 [myid:] - INFO  [WorkerReceiver[myid=4]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,482 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@688] - Notification: 2 (message format version), 1 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,484 [myid:] - INFO  [/127.0.0.1:11238:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:43255
2016-07-21 04:24:14,485 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@915] - Connection broken for id 1, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,485 [myid:] - INFO  [/127.0.0.1:11232:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:45444
2016-07-21 04:24:14,485 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,486 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@915] - Connection broken for id 1, my id = 3, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,486 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,487 [myid:] - INFO  [WorkerSender[myid=5]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (6, 5)
2016-07-21 04:24:14,487 [myid:] - INFO  [WorkerSender[myid=5]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (7, 5)
2016-07-21 04:24:14,489 [myid:] - INFO  [WorkerReceiver[myid=5]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,490 [myid:] - INFO  [WorkerSender[myid=5]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (8, 5)
2016-07-21 04:24:14,490 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,490 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 1 my id = 3
2016-07-21 04:24:14,490 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,490 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 1 my id = 5
2016-07-21 04:24:14,491 [myid:] - INFO  [/127.0.0.1:11241:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:44375
2016-07-21 04:24:14,491 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@688] - Notification: 2 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,491 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@915] - Connection broken for id 1, my id = 6, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,492 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,492 [myid:] - INFO  [/127.0.0.1:11244:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:40447
2016-07-21 04:24:14,493 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@915] - Connection broken for id 1, my id = 7, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,493 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,495 [myid:] - INFO  [/127.0.0.1:11235:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:59547
2016-07-21 04:24:14,497 [myid:] - INFO  [/127.0.0.1:11229:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:60301
2016-07-21 04:24:14,498 [myid:] - INFO  [/127.0.0.1:11247:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:57718
2016-07-21 04:24:14,498 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@915] - Connection broken for id 1, my id = 2, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,499 [myid:] - WARN  [RecvWorker:1:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,500 [myid:] - INFO  [/127.0.0.1:11247:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:57732
2016-07-21 04:24:14,500 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,500 [myid:] - INFO  [WorkerSender[myid=4]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (5, 4)
2016-07-21 04:24:14,501 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 1 my id = 7
2016-07-21 04:24:14,501 [myid:] - INFO  [WorkerSender[myid=4]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (6, 4)
2016-07-21 04:24:14,502 [myid:] - INFO  [WorkerSender[myid=4]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (7, 4)
2016-07-21 04:24:14,502 [myid:] - INFO  [WorkerSender[myid=4]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (8, 4)
2016-07-21 04:24:14,504 [myid:] - INFO  [WorkerReceiver[myid=4]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,504 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@688] - Notification: 2 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,505 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,505 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 1 my id = 6
2016-07-21 04:24:14,505 [myid:] - INFO  [/127.0.0.1:11238:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:43280
2016-07-21 04:24:14,506 [myid:] - INFO  [/127.0.0.1:11223:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:42457
2016-07-21 04:24:14,506 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@915] - Connection broken for id 4, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,506 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,507 [myid:] - INFO  [/127.0.0.1:11226:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:51083
2016-07-21 04:24:14,507 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@837] - Exception when using channel: for id 3 my id = 1 error = java.net.SocketException: Broken pipe
2016-07-21 04:24:14,507 [myid:] - WARN  [RecvWorker:3:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,508 [myid:] - INFO  [/127.0.0.1:11226:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:51084
2016-07-21 04:24:14,508 [myid:] - INFO  [WorkerReceiver[myid=0]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,508 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@837] - Exception when using channel: for id 6 my id = 1 error = java.net.SocketException: Broken pipe
2016-07-21 04:24:14,508 [myid:] - INFO  [WorkerReceiver[myid=0]:FastLeaderElection@688] - Notification: 2 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,508 [myid:] - INFO  [WorkerReceiver[myid=5]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,509 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@688] - Notification: 2 (message format version), 0 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 0 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,510 [myid:] - INFO  [WorkerReceiver[myid=1]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,510 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection@688] - Notification: 2 (message format version), 6 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 6 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,510 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,516 [myid:] - INFO  [WorkerSender[myid=2]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (3, 2)
2016-07-21 04:24:14,516 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:14,517 [myid:] - WARN  [SendWorker:1:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 1 my id = 2
2016-07-21 04:24:14,517 [myid:] - INFO  [WorkerReceiver[myid=2]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,517 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@837] - Exception when using channel: for id 5 my id = 4 error = java.net.SocketException: Broken pipe
2016-07-21 04:24:14,518 [myid:] - INFO  [/127.0.0.1:11235:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:59564
2016-07-21 04:24:14,518 [myid:] - INFO  [WorkerSender[myid=2]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (4, 2)
2016-07-21 04:24:14,518 [myid:] - INFO  [WorkerReceiver[myid=2]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,518 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:14,520 [myid:] - INFO  [/127.0.0.1:11232:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:45462
2016-07-21 04:24:14,520 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 6 my id = 1
2016-07-21 04:24:14,520 [myid:] - INFO  [/127.0.0.1:11232:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:45470
2016-07-21 04:24:14,521 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 3 my id = 1
2016-07-21 04:24:14,521 [myid:] - INFO  [WorkerReceiver[myid=3]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:14,521 [myid:] - INFO  [/127.0.0.1:11226:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:51085
2016-07-21 04:24:14,522 [myid:] - INFO  [WorkerSender[myid=3]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (4, 3)
2016-07-21 04:24:14,522 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection@688] - Notification: 2 (message format version), 3 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:14,522 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@915] - Connection broken for id 5, my id = 1, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:14,522 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@837] - Exception when using channel: for id 5 my id = 1 error = java.net.SocketException: Broken pipe
2016-07-21 04:24:14,523 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 5 my id = 1
2016-07-21 04:24:14,522 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zoo
...[truncated 231211 chars]...
.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,861 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 3 my id = 6
2016-07-21 04:24:40,805 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 4 my id = 3
2016-07-21 04:24:40,805 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,805 [myid:] - WARN  [SendWorker:3:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 3 my id = 7
2016-07-21 04:24:40,804 [myid:] - INFO  [WorkerReceiver[myid=6]:FastLeaderElection@688] - Notification: 2 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,804 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@688] - Notification: 2 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,862 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,861 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), LEADING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,860 [myid:] - INFO  [/127.0.0.1:11235:QuorumCnxManager$Listener@661] - Leaving listener
2016-07-21 04:24:40,859 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,863 [myid:] - INFO  [main:QuorumBase@398] - Shutting down leader election QuorumPeer
2016-07-21 04:24:40,863 [myid:] - INFO  [main:QuorumBase@403] - Waiting for QuorumPeer to exit thread
2016-07-21 04:24:40,863 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@915] - Connection broken for id 4, my id = 7, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@915] - Connection broken for id 5, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,864 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@837] - Exception when using channel: for id 4 my id = 7 error = java.net.SocketException: Broken pipe
2016-07-21 04:24:40,865 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,862 [myid:] - INFO  [WorkerReceiver[myid=4]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,862 [myid:] - INFO  [WorkerReceiver[myid=6]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,865 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), LEADING (n.state), 2 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,865 [myid:] - INFO  [main:QuorumBase@394] - Shutting down quorum peer QuorumPeer
2016-07-21 04:24:40,865 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection@688] - Notification: 2 (message format version), 5 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 5 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,865 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 4 my id = 7
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@915] - Connection broken for id 7, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,866 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,864 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@915] - Connection broken for id 4, my id = 5, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,867 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,864 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,864 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 7 my id = 4
2016-07-21 04:24:40,864 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@915] - Connection broken for id 8, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,868 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,863 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@915] - Connection broken for id 4, my id = 6, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,869 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,863 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@915] - Connection broken for id 4, my id = 8, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,870 [myid:] - WARN  [RecvWorker:4:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,863 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,863 [myid:] - INFO  [WorkerReceiver[myid=5]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,863 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,863 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@915] - Connection broken for id 6, my id = 4, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,871 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,871 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 6 my id = 4
2016-07-21 04:24:40,871 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@688] - Notification: 2 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,870 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 8 my id = 4
2016-07-21 04:24:40,870 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,869 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,868 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 5 my id = 4
2016-07-21 04:24:40,868 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,867 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 8 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 8 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,866 [myid:] - INFO  [/127.0.0.1:11238:QuorumCnxManager$Listener@661] - Leaving listener
2016-07-21 04:24:40,866 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,866 [myid:] - INFO  [WorkerReceiver[myid=4]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:40,874 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,874 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 8 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 8 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,866 [myid:] - INFO  [WorkerReceiver[myid=6]:FastLeaderElection@688] - Notification: 2 (message format version), 6 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 6 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,866 [myid:] - INFO  [Thread-5:FLEZeroWeightTest$LEThread@115] - Thread 4 got a null vote
2016-07-21 04:24:40,875 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,875 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), FOLLOWING (n.state), 1 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,873 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 4 my id = 5
2016-07-21 04:24:40,875 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,875 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), FOLLOWING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,872 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 4 my id = 6
2016-07-21 04:24:40,872 [myid:] - INFO  [WorkerReceiver[myid=5]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,876 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,876 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 8 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 8 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,876 [myid:] - WARN  [WorkerSender[myid=5]:QuorumCnxManager@455] - Cannot open channel to 4 at election address /127.0.0.1:11235
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:441)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:482)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:419)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:486)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:465)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@915] - Connection broken for id 7, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,872 [myid:] - WARN  [SendWorker:4:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 4 my id = 8
2016-07-21 04:24:40,877 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), FOLLOWING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@915] - Connection broken for id 5, my id = 8, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@915] - Connection broken for id 5, my id = 6, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,877 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@915] - Connection broken for id 8, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,879 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@915] - Connection broken for id 5, my id = 7, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,877 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,877 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,877 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@915] - Connection broken for id 6, my id = 5, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,881 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,877 [myid:] - INFO  [WorkerSender[myid=5]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:40,877 [myid:] - INFO  [main:QuorumBase@398] - Shutting down leader election QuorumPeer
2016-07-21 04:24:40,876 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,876 [myid:] - INFO  [Thread-9:FLEZeroWeightTest$LEThread@125] - Finished election: 8, 2
2016-07-21 04:24:40,876 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 6 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 6 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,875 [myid:] - INFO  [WorkerReceiver[myid=6]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,881 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 6 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 6 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,881 [myid:] - INFO  [main:QuorumBase@403] - Waiting for QuorumPeer to exit thread
2016-07-21 04:24:40,880 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 6 my id = 5
2016-07-21 04:24:40,880 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 7 my id = 5
2016-07-21 04:24:40,880 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,879 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 8 my id = 5
2016-07-21 04:24:40,878 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,883 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,878 [myid:] - WARN  [RecvWorker:5:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,878 [myid:] - INFO  [WorkerReceiver[myid=5]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:40,883 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,883 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,883 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 5 my id = 7
2016-07-21 04:24:40,883 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,882 [myid:] - INFO  [main:QuorumBase@394] - Shutting down quorum peer QuorumPeer
2016-07-21 04:24:40,882 [myid:] - INFO  [WorkerReceiver[myid=6]:FastLeaderElection@688] - Notification: 2 (message format version), 7 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 7 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,884 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 5 my id = 6
2016-07-21 04:24:40,884 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 8 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 8 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,884 [myid:] - WARN  [SendWorker:5:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 5 my id = 8
2016-07-21 04:24:40,883 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,889 [myid:] - INFO  [WorkerReceiver[myid=6]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,885 [myid:] - INFO  [/127.0.0.1:11241:QuorumCnxManager$Listener@661] - Leaving listener
2016-07-21 04:24:40,890 [myid:] - INFO  [Thread-7:FLEZeroWeightTest$LEThread@115] - Thread 6 got a null vote
2016-07-21 04:24:40,891 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,891 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 7 my id = 6
2016-07-21 04:24:40,890 [myid:] - INFO  [WorkerReceiver[myid=6]:FastLeaderElection@688] - Notification: 2 (message format version), 7 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 7 (n.sid), 0x0 (n.peerEPoch), LOOKING (my state)0 (n.config version)
2016-07-21 04:24:40,889 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), FOLLOWING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,893 [myid:] - INFO  [WorkerReceiver[myid=6]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:40,892 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,892 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@915] - Connection broken for id 6, my id = 7, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,894 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,891 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@915] - Connection broken for id 6, my id = 8, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,894 [myid:] - WARN  [RecvWorker:6:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,891 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@915] - Connection broken for id 8, my id = 6, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,894 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,891 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@915] - Connection broken for id 7, my id = 6, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:40,895 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,891 [myid:] - INFO  [main:QuorumBase@398] - Shutting down leader election QuorumPeer
2016-07-21 04:24:40,891 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,895 [myid:] - INFO  [main:QuorumBase@403] - Waiting for QuorumPeer to exit thread
2016-07-21 04:24:40,894 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,894 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,894 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:40,896 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 6 my id = 7
2016-07-21 04:24:40,893 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 8 my id = 6
2016-07-21 04:24:40,896 [myid:] - WARN  [SendWorker:6:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 6 my id = 8
2016-07-21 04:24:40,896 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), FOLLOWING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,896 [myid:] - INFO  [main:QuorumBase@394] - Shutting down quorum peer QuorumPeer
2016-07-21 04:24:40,896 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 8 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 8 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:40,897 [myid:] - INFO  [/127.0.0.1:11244:QuorumCnxManager$Listener@661] - Leaving listener
2016-07-21 04:24:40,897 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,898 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@915] - Connection broken for id 7, my id = 8, error = 
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:392)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:47,925 [myid:] - WARN  [RecvWorker:7:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,898 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@915] - Connection broken for id 8, my id = 7, error = 
java.net.SocketException: Socket closed
	at java.net.SocketInputStream.socketRead0(Native Method)
	at java.net.SocketInputStream.read(SocketInputStream.java:152)
	at java.net.SocketInputStream.read(SocketInputStream.java:122)
	at java.net.SocketInputStream.read(SocketInputStream.java:210)
	at java.io.DataInputStream.readInt(DataInputStream.java:387)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$RecvWorker.run(QuorumCnxManager.java:900)
2016-07-21 04:24:47,926 [myid:] - WARN  [RecvWorker:8:QuorumCnxManager$RecvWorker@918] - Interrupting SendWorker
2016-07-21 04:24:40,898 [myid:] - INFO  [main:QuorumBase@398] - Shutting down leader election QuorumPeer
2016-07-21 04:24:43,790 [myid:] - INFO  [WorkerSender[myid=3]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:43,790 [myid:] - INFO  [WorkerReceiver[myid=3]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:43,642 [myid:] - INFO  [WorkerSender[myid=1]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:43,642 [myid:] - INFO  [WorkerReceiver[myid=1]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:43,585 [myid:] - INFO  [WorkerSender[myid=4]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:43,574 [myid:] - INFO  [WorkerSender[myid=6]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:40,898 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@837] - Exception when using channel: for id 8 my id = 7 error = java.net.SocketException: Socket closed
2016-07-21 04:24:40,898 [myid:] - INFO  [WorkerReceiver[myid=7]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:40,898 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 4 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 4 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:47,927 [myid:] - WARN  [SendWorker:8:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 8 my id = 7
2016-07-21 04:24:47,926 [myid:] - INFO  [main:QuorumBase@403] - Waiting for QuorumPeer to exit thread
2016-07-21 04:24:47,926 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@832] - Interrupted while waiting for message on queue
java.lang.InterruptedException
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2017)
	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2095)
	at java.util.concurrent.ArrayBlockingQueue.poll(ArrayBlockingQueue.java:389)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.pollSendQueue(QuorumCnxManager.java:982)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.access$500(QuorumCnxManager.java:63)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$SendWorker.run(QuorumCnxManager.java:820)
2016-07-21 04:24:47,928 [myid:] - INFO  [/127.0.0.1:11247:QuorumCnxManager$Listener@638] - Received connection request /127.0.0.1:57823
2016-07-21 04:24:47,928 [myid:] - INFO  [WorkerSender[myid=7]:QuorumCnxManager@276] - Have smaller server identifier, so dropping the connection: (8, 7)
2016-07-21 04:24:47,928 [myid:] - INFO  [WorkerReceiver[myid=8]:QuorumHierarchical@136] - 9, 9, 3
2016-07-21 04:24:47,928 [myid:] - WARN  [SendWorker:7:QuorumCnxManager$SendWorker@841] - Send worker leaving thread  id 7 my id = 8
2016-07-21 04:24:47,929 [myid:] - INFO  [main:QuorumBase@394] - Shutting down quorum peer QuorumPeer
2016-07-21 04:24:47,929 [myid:] - WARN  [WorkerSender[myid=8]:QuorumCnxManager@455] - Cannot open channel to 4 at election address /127.0.0.1:11235
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:441)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:482)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:419)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:486)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:465)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 04:24:47,929 [myid:] - INFO  [WorkerSender[myid=7]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:47,929 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection@688] - Notification: 2 (message format version), 2 (n.leader), 0x0 (n.zxid), 0xffffffffffffffff (n.round), FOLLOWING (n.state), 3 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:47,930 [myid:] - INFO  [WorkerReceiver[myid=7]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:47,930 [myid:] - WARN  [/127.0.0.1:11247:QuorumCnxManager@455] - Cannot open channel to 7 at election address /127.0.0.1:11244
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:441)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.receiveConnection(QuorumCnxManager.java:367)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager$Listener.run(QuorumCnxManager.java:640)
2016-07-21 04:24:47,930 [myid:] - INFO  [/127.0.0.1:11247:QuorumCnxManager$Listener@661] - Leaving listener
2016-07-21 04:24:47,930 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection@688] - Notification: 2 (message format version), 7 (n.leader), 0x0 (n.zxid), 0x1 (n.round), LOOKING (n.state), 7 (n.sid), 0x0 (n.peerEPoch), FOLLOWING (my state)0 (n.config version)
2016-07-21 04:24:47,931 [myid:] - INFO  [main:QuorumBase@398] - Shutting down leader election QuorumPeer
2016-07-21 04:24:47,931 [myid:] - INFO  [main:QuorumBase@403] - Waiting for QuorumPeer to exit thread
2016-07-21 04:24:47,931 [myid:] - INFO  [WorkerReceiver[myid=8]:FastLeaderElection$Messenger$WorkerReceiver@440] - WorkerReceiver is down
2016-07-21 04:24:47,932 [myid:] - WARN  [WorkerSender[myid=8]:QuorumCnxManager@455] - Cannot open channel to 7 at election address /127.0.0.1:11244
java.net.ConnectException: Connection refused
	at java.net.PlainSocketImpl.socketConnect(Native Method)
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
	at java.net.Socket.connect(Socket.java:579)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:441)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.connectOne(QuorumCnxManager.java:482)
	at org.apache.zookeeper.server.quorum.QuorumCnxManager.toSend(QuorumCnxManager.java:419)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.process(FastLeaderElection.java:486)
	at org.apache.zookeeper.server.quorum.FastLeaderElection$Messenger$WorkerSender.run(FastLeaderElection.java:465)
	at java.lang.Thread.run(Thread.java:745)
2016-07-21 04:24:47,932 [myid:] - INFO  [WorkerSender[myid=8]:FastLeaderElection$Messenger$WorkerSender@470] - WorkerSender is down
2016-07-21 04:24:47,933 [myid:] - INFO  [main:ZKTestCase$1@70] - FAILED testZeroWeightQuorum
java.lang.AssertionError: Threads didn't join
	at org.junit.Assert.fail(Assert.java:88)
	at org.apache.zookeeper.test.FLEZeroWeightTest.testZeroWeightQuorum(FLEZeroWeightTest.java:167)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.apache.zookeeper.JUnit4ZKTestRunner$LoggedInvokeMethod.evaluate(JUnit4ZKTestRunner.java:79)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.junit.rules.TestWatchman$1.evaluate(TestWatchman.java:53)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at junit.framework.JUnit4TestAdapter.run(JUnit4TestAdapter.java:38)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
	at org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)
2016-07-21 04:24:47,933 [myid:] - INFO  [main:ZKTestCase$1@60] - FINISHED testZeroWeightQuorum

Attachments

Issue Links

relates to

ZOOKEEPER-2135 fix trunk build

Open

links to

GitHub Pull Request #88

GitHub Pull Request #89

GitHub Pull Request #93

Activity

People

Assignee:: Andor Molnar

Reporter:: Michael Han

Votes:: 0 Vote for this issue

Watchers:: 4 Start watching this issue

Dates

Created:: 21/Jul/16 21:33

Updated:: 19/Dec/19 23:02

Resolved:: 25/Oct/18 15:11