Description
We're seeing the following errors in our logs from time to time (about once an hour):
ERROR [MutationStage:10] 2014-01-19 10:36:26,659 CassandraDaemon.java (line 187) Exception in thread Thread[MutationStage:10,5,main] java.lang.AssertionError: Verb COUNTER_MUTATION should not legally be dropped at org.apache.cassandra.net.MessagingService.incrementDroppedMessages(MessagingService.java:779) at org.apache.cassandra.service.StorageProxy$DroppableRunnable.run(StorageProxy.java:1925) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:724)
They always appear in groups. About 50-100 errors in a row.
We've got a 12-nodes cluster recently upgraded from 1.2.10 to 2.0.4. It's under pretty heavy load.