[CASSANDRA-12942] ClassCastException during Status

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Cannot Reproduce
• Fix Version/s: 3.0.x, 3.11.x
• Component/s: Legacy/Tools

Environment

Cassandra 3.7
OpenJDK 8
Ubuntu 14.04

• Severity: Low
• Assignee: Stefan Miklosovic
• Reporter: Chris Donati

Authors

Stefan Miklosovic

• Created: 22/Nov/16 05:08
• Updated: 02/Dec/22 14:24
• Resolved: 02/Dec/22 14:24

Description

I often encounter a ClassCastException when trying to run `nodetool status` on a particular cluster. Occasionally, the command will work on one of the nodes (and report all of the nodes as 'UN'), but the majority of the time, nodetool raises the following exception:

error: null
-- StackTrace --
java.lang.ClassCastException

A couple of times, I've gotten lucky and nodetool has provided a more verbose error message:

error: org.apache.cassandra.dht.LocalPartitioner$LocalToken cannot be cast to org.apache.cassandra.dht.ByteOrderedPartitioner$BytesToken
-- StackTrace --
java.lang.ClassCastException: org.apache.cassandra.dht.LocalPartitioner$LocalToken cannot be cast to org.apache.cassandra.dht.ByteOrderedPartitioner$BytesToken
	at org.apache.cassandra.dht.ByteOrderedPartitioner$BytesToken.compareTo(ByteOrderedPartitioner.java:79)
	at org.apache.cassandra.dht.ByteOrderedPartitioner$BytesToken.compareTo(ByteOrderedPartitioner.java:55)
	at org.apache.cassandra.dht.Token$KeyBound.compareTo(Token.java:166)
	at org.apache.cassandra.dht.Token$KeyBound.compareTo(Token.java:145)
	at org.apache.cassandra.db.DecoratedKey.compareTo(DecoratedKey.java:93)
	at org.apache.cassandra.io.sstable.IndexSummary.binarySearch(IndexSummary.java:122)
	at org.apache.cassandra.io.sstable.format.SSTableReader.getSampleIndexesForRanges(SSTableReader.java:1345)
	at org.apache.cassandra.io.sstable.format.SSTableReader.getKeySamples(SSTableReader.java:1379)
	at org.apache.cassandra.db.ColumnFamilyStore.keySamples(ColumnFamilyStore.java:2058)
	at org.apache.cassandra.service.StorageService.keySamples(StorageService.java:3722)
	at org.apache.cassandra.service.StorageService.getSplits(StorageService.java:3678)
	at org.apache.cassandra.dht.ByteOrderedPartitioner.describeOwnership(ByteOrderedPartitioner.java:284)
	at org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:4460)
	at org.apache.cassandra.service.StorageService.effectiveOwnership(StorageService.java:184)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
	at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
	at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
	at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:252)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:819)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:801)
	at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1468)
	at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
	at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1309)
	at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1401)
	at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:829)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:324)
	at sun.rmi.transport.Transport$1.run(Transport.java:200)
	at sun.rmi.transport.Transport$1.run(Transport.java:197)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:196)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:568)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:826)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.lambda$run$0(TCPTransport.java:683)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:682)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

There are 3 nodes in the cluster, and it's using the SimpleStrategy with a 3x replication factor. It's using the ByteOrderedPartitioner (which I know is frowned upon).