[CASSANDRA-15472] Read failure due to exception from metrics-core dependency

• Type: Bug
• Status: In Progress
• Priority: Normal
• Resolution: Unresolved
• Fix Version/s: 3.0.x, 3.11.x, 4.0.x
• Component/s: Dependencies
• Bug Category: Availability - Response Crash
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Assignee: Sumanth Pasupuleti
• Reporter: Sumanth Pasupuleti

Authors

Sumanth Pasupuleti

• Created: 06/Jan/20 21:13
• Updated: 03/May/23 08:17

Description

Stacktrace

Uncaught exception on thread Thread[SharedPool-Worker-27,5,main]: {}
java.util.NoSuchElementException: null
	at java.util.concurrent.ConcurrentSkipListMap.firstKey(ConcurrentSkipListMap.java:2053) ~[na:1.8.0_222]
	at com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:102) ~[metrics-core-2.2.0.jar:na]
	at com.yammer.metrics.stats.ExponentiallyDecayingSample.update(ExponentiallyDecayingSample.java:81) ~[metrics-core-2.2.0.jar:na]
	at com.yammer.metrics.core.Histogram.update(Histogram.java:110) ~[metrics-core-2.2.0.jar:na]
	at com.yammer.metrics.core.Timer.update(Timer.java:198) ~[metrics-core-2.2.0.jar:na]
	at com.yammer.metrics.core.Timer.update(Timer.java:76) ~[metrics-core-2.2.0.jar:na]
	at org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:108) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.metrics.LatencyMetrics.addNano(LatencyMetrics.java:114) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1897) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:353) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:85) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.db.ReadVerbHandler.doVerb(ReadVerbHandler.java:47) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:64) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_222]
	at org.apache.cassandra.concurrent.AbstractTracingAwareExecutorService$FutureTask.run(AbstractTracingAwareExecutorService.java:164) ~[nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:105) [nf-cassandra-2.1.19.10.jar:2.1.19.10]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_222]

This issue has been fixed in v4.0.6.

This is observed on a 2.1.19 cluster, but this would impact pretty much any version of C* since we depend on lower versions of metrics-core that do not have the fix.

Issue Links

• is related to: CASSANDRA-14667 Upgrade Dropwizard Metrics to 4.x, logback to 1.2.12 and cassandra-driver-3.11.5