[CASSANDRA-8450] Cassandra node announces availability before it is ready to accept CQL clients

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Duplicate
• Severity: Normal
• Since Version: 2.0.9
• Assignee: Unassigned
• Reporter: Vishy Kasar
• Created: 09/Dec/14 22:16
• Updated: 16/Apr/19 09:31
• Resolved: 09/Dec/14 22:42

Description

We see cases where on a restart, the cassandra node announces the availability to java driver clients before it is ready to accept CQL clients. The java-driver gets an onUp message, attempts to create a connection to the cassandra node which promptly rejects its connection. When there are large number of clients, this error is seen by all clients. We should delay the announcements until the node is ready to accept connections.

server log:
INFO [main] 2014-12-07 01:34:47,921 Server.java (line 156) Starting listening for CQL clients on host/ip:9042...

client log:
com.datastax.driver.core.TransportException: [host1/ip1:9042] Cannot connect
at com.datastax.driver.core.Connection.<init>(Connection.java:105)
at com.datastax.driver.core.PooledConnection.<init>(PooledConnection.java:35)
at com.datastax.driver.core.Connection$Factory.open(Connection.java:522)
at com.datastax.driver.core.HostConnectionPool.<init>(HostConnectionPool.java:86)
at com.datastax.driver.core.SessionManager.replacePool(SessionManager.java:269)
at com.datastax.driver.core.SessionManager.access$400(SessionManager.java:39)
at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:301)
at com.datastax.driver.core.SessionManager$3.call(SessionManager.java:293)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: Connection refused: host1/ip1:9042
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:739)
at org.jboss.netty.channel.socket.nio.NioClientBoss.connect(NioClientBoss.java:150)
at org.jboss.netty.channel.socket.nio.NioClientBoss.processSelectedKeys(NioClientBoss.java:105)
at org.jboss.netty.channel.socket.nio.NioClientBoss.process(NioClientBoss.java:79)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at org.jboss.netty.channel.socket.nio.NioClientBoss.run(NioClientBoss.java:42)
... 3 more
2014-12-07 01:34:44,011 [Cassandra Java Driver worker-1070] - onUp address=ip1 host=host1

Issue Links

• duplicates: CASSANDRA-8236 Delay "node up" and "node added" notifications until native protocol server is started