[CASSANDRA-9625] GraphiteReporter not reporting

• Type: Bug
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Fix Version/s: 3.0.x, 3.11.x, 5.x
• Component/s: Observability/Metrics
• Labels: remove-reopen

Environment

Debian Jessie, 7u79-2.5.5-1~deb8u1, Cassandra 2.1.3

• Severity: Normal
• Assignee: Unassigned
• Reporter: Eric Evans

Reviewers

Marcus Eriksson

• Created: 19/Jun/15 18:40
• Updated: 07/Mar/23 10:53

Description

When upgrading from 2.1.3 to 2.1.6, the Graphite metrics reporter stops working. The usual startup is logged, and one batch of samples is sent, but the reporting interval comes and goes, and no other samples are ever sent. The logs are free from errors.

Frustratingly, metrics reporting works in our smaller (staging) environment on 2.1.6; We are able to reproduce this on all 6 of production nodes, but not on a 3 node (otherwise identical) staging cluster (maybe it takes a certain level of concurrency?).

Attached is a thread dump, and our metrics.yaml.

Issue Links

• is related to: CASSANDRA-10676 AssertionError in CompactionExecutor
• relates to: CASSANDRA-11599 When there are a large number of small repaired L0 sstables, compaction is very slow