Details
-
Bug
-
Status: Resolved
-
Low
-
Resolution: Cannot Reproduce
-
None
-
None
-
2-node cluster on ec2, ubuntu, cassandra-1.2.0 commit a32eb9f7d2f2868e8154d178e96e045859e1d855
-
Low
Description
Start both nodes, start stress on one node like this: "cassandra-stress --replication-factor=2 --operation=COUNTER_ADD"
While that is running: On the other node, kill cassandra, wait for "nodetool status" to show the node as down, and restart cassandra. I sometimes have to kill and restart cassandra several times to get the problem to happen.
I get this error several times in the log:
ERROR 15:39:33,198 Exception in thread Thread[MutationStage:16,5,main] java.lang.AssertionError at org.apache.cassandra.locator.TokenMetadata.firstTokenIndex(TokenMetadata.java:748) at org.apache.cassandra.locator.TokenMetadata.firstToken(TokenMetadata.java:762) at org.apache.cassandra.locator.AbstractReplicationStrategy.getNaturalEndpoints(AbstractReplicationStrategy.java:95) at org.apache.cassandra.service.StorageService.getNaturalEndpoints(StorageService.java:2426) at org.apache.cassandra.service.StorageProxy.performWrite(StorageProxy.java:396) at org.apache.cassandra.service.StorageProxy.applyCounterMutationOnLeader(StorageProxy.java:755) at org.apache.cassandra.db.CounterMutationVerbHandler.doVerb(CounterMutationVerbHandler.java:53) at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:56) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)