Details
-
Bug
-
Status: Resolved
-
Low
-
Resolution: Cannot Reproduce
-
None
-
None
-
12 node ring of 1.2.x.
11 of 12 are 1.2.15.
1 is 1.2.16.
-
Low
Description
12 Cassandra instances, one per node.
11 of the Cassandra instances are 1.2.15.
1 of the Cassandra instances is 1.2.16.
One of the eleven 1.2.15 Cassandra instances is restarted (disable thrift, gossip, then flush, drain, stop, start).
The 1.2.16 Cassandra instance noted this by throwing a Null Pointer Exception. None of the 1.2.15 instances threw an exception and this is new behavior that hasn't been observed before.
ERROR 02:18:06,009 Exception in thread Thread[GossipStage:1,5,main]
java.lang.NullPointerException
at org.apache.cassandra.gms.Gossiper.convict(Gossiper.java:264)
at org.apache.cassandra.gms.FailureDetector.forceConviction(FailureDetector.java:246)
at org.apache.cassandra.gms.GossipShutdownVerbHandler.doVerb(GossipShutdownVerbHandler.java:37)
at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
INFO 02:18:23,402 Node /10.x.y.x is now part of the cluster
INFO 02:18:23,403 InetAddress /10.x.y.z is now UP
INFO 02:18:53,494 FatClient /10.x.y.z has been silent for 30000ms, removing from gossip
INFO 02:19:00,031 Handshaking version with /10.x.y.z