[CASSANDRA-11823] Creating a table leads to a race with GraphiteReporter

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 3.0.9, 3.8
• Component/s: Legacy/Observability
• Labels: lhf
• Severity: Low
• Since Version: 3.0.6
• Assignee: Edward Ribeiro
• Reporter: Stefano Ortolani

Authors

Edward Ribeiro

Reviewers

Stefania Alborghetti

• Created: 18/May/16 00:51
• Updated: 16/Apr/19 09:30
• Resolved: 04/Aug/16 00:55

Description

Happened only on 3/4 nodes out of 13.

INFO  [MigrationStage:1] 2016-05-18 00:34:11,566 ColumnFamilyStore.java:381 - Initializing schema.table
ERROR [metrics-graphite-reporter-1-thread-1] 2016-05-18 00:34:11,569 ScheduledReporter.java:119 - RuntimeException thrown from GraphiteReporter#report. Exception was suppressed.
java.util.ConcurrentModificationException: null
	at java.util.HashMap$HashIterator.nextNode(HashMap.java:1429) ~[na:1.8.0_91]
	at java.util.HashMap$KeyIterator.next(HashMap.java:1453) ~[na:1.8.0_91]
	at org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:690) ~[apache-cassandra-3.0.6.jar:3.0.6]
	at org.apache.cassandra.metrics.TableMetrics$33.getValue(TableMetrics.java:686) ~[apache-cassandra-3.0.6.jar:3.0.6]
	at com.codahale.metrics.graphite.GraphiteReporter.reportGauge(GraphiteReporter.java:281) ~[metrics-graphite-3.1.0.jar:3.1.0]
	at com.codahale.metrics.graphite.GraphiteReporter.report(GraphiteReporter.java:158) ~[metrics-graphite-3.1.0.jar:3.1.0]
	at com.codahale.metrics.ScheduledReporter.report(ScheduledReporter.java:162) ~[metrics-core-3.1.0.jar:3.1.0]
	at com.codahale.metrics.ScheduledReporter$1.run(ScheduledReporter.java:117) ~[metrics-core-3.1.0.jar:3.1.0]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_91]
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_91]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_91]
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_91]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_91]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_91]
	at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]