[CASSANDRA-9580] Cardinality check broken during incremental compaction re-opening

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 2.1.7
• Component/s: Local/Compaction
• Severity: Low
• Assignee: T Jake Luciani
• Reporter: T Jake Luciani

Authors

T Jake Luciani

Reviewers

Benedict Elliott Smith

• Created: 11/Jun/15 15:07
• Updated: 16/Apr/19 09:31
• Resolved: 18/Jun/15 18:49

Description

While testing LCS I found cfstats sometimes crashes during compaction

It looks to be related to the incremental re-opening not having metadata.

----------------
Keyspace: stresscql
	Read Count: 0
	Read Latency: NaN ms.
	Write Count: 6590571
	Write Latency: 0.026910956273743198 ms.
	Pending Flushes: 0
		Table: ycsb
		SSTable count: 69
		SSTables in each level: [67/4, 1, 0, 0, 0, 0, 0, 0, 0]
		Space used (live): 3454857914
		Space used (total): 3454857914
		Space used by snapshots (total): 0
		Off heap memory used (total): 287361
		SSTable Compression Ratio: 0.0
error: /home/jake/workspace/cassandra/./bin/../data/data/stresscql/ycsb-ff399910104911e5a797a18c989fb6f2/stresscql-ycsb-tmplink-ka-125-Data.db
-- StackTrace --
java.lang.AssertionError: /home/jake/workspace/cassandra/./bin/../data/data/stresscql/ycsb-ff399910104911e5a797a18c989fb6f2/stresscql-ycsb-tmplink-ka-125-Data.db
	at org.apache.cassandra.io.sstable.SSTableReader.getApproximateKeyCount(SSTableReader.java:270)
	at org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:296)
	at org.apache.cassandra.metrics.ColumnFamilyMetrics$9.value(ColumnFamilyMetrics.java:290)
	at com.yammer.metrics.reporting.JmxReporter$Gauge.getValue(JmxReporter.java:63)
	at sun.reflect.GeneratedMethodAccessor10.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
	at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:112)
	at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:46)
	at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:237)
	at com.sun.jmx.mbeanserver.PerInterface.getAttribute(PerInterface.java:83)
	at com.sun.jmx.mbeanserver.MBeanSupport.getAttribute(MBeanSupport.java:206)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getAttribute(DefaultMBeanServerInterceptor.java:647)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.getAttribute(JmxMBeanServer.java:678)
	at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1443)
	at javax.management.remote.rmi.RMIConnectionImpl.access$300(RMIConnectionImpl.java:76)
	at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1307)
	at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1399)
	at javax.management.remote.rmi.RMIConnectionImpl.getAttribute(RMIConnectionImpl.java:637)
	at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:483)
	at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:323)
	at sun.rmi.transport.Transport$1.run(Transport.java:178)
	at sun.rmi.transport.Transport$1.run(Transport.java:175)
	at java.security.AccessController.doPrivileged(Native Method)
	at sun.rmi.transport.Transport.serviceCall(Transport.java:174)
	at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:557)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:812)
	at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:671)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Issue Links

• is duplicated by: CASSANDRA-9612 Assertion error while running `nodetool cfstats`
• is duplicated by: CASSANDRA-9616 cfstats on 2.1.6 throws fatal exception during compaction cycles