[CASSANDRA-11323] When node runs out of commitlog space you get poor log information

• Type: Bug
• Status: Resolved
• Priority: Low
• Resolution: Fixed
• Fix Version/s: 3.0.26, 3.11.12, 4.0.2
• Component/s: Legacy/Observability
• Labels: fallout
• Severity: Low
• Since Version: 3.0.0

Source Control Link

https://github.com/apache/cassandra/commit/84eb1b8c85371e1d73bb8a0e60393de8bf4af28b

• Assignee: Boris Onufriyev
• Reporter: T Jake Luciani

Authors

Boris Onufriyev

Reviewers

Ariel Weisberg, Benjamin Lerer, Brandon Williams

• Created: 09/Mar/16 15:31
• Updated: 16/Mar/22 16:36
• Resolved: 14/Oct/21 14:49

Description

ERROR [PERIODIC-COMMIT-LOG-SYNCER] 2016-03-08 20:27:33,899 StorageService.java:470 - Stopping gossiper
WARN  [PERIODIC-COMMIT-LOG-SYNCER] 2016-03-08 20:27:33,899 StorageService.java:377 - Stopping gossip by operator request
INFO  [PERIODIC-COMMIT-LOG-SYNCER] 2016-03-08 20:27:33,899 Gossiper.java:1463 - Announcing shutdown

That's all you get when a node runs out of commit log space.

We should explicitly callout the fact the commitlog is out of disk. I see that in the commit log error handler but after it shuts down. So I think it's never getting written before shutdown.