[CASSANDRA-15547] commit logs fill up disk space

• Type: Bug
• Status: Triage Needed
• Priority: Normal
• Resolution: Unresolved
• Component/s: Local/Commit Log
• Platform: All
• Assignee: Unassigned
• Reporter: chongsheng yu
• Created: 05/Feb/20 13:31
• Updated: 05/Feb/20 13:31

Description

We have a three-nodes Cassandra cluster with version 3.11.3 in our production environment and the configuration 'commitlog_total_space_in_mb' is set to 8192.

 

After tens of days running, commit logs at the third node grow upto about 300G and run out of disk space finally. The amount of commitlog files is about 10K. We have to remove all commitlog files at this node and restart Cassandra to repair it.

 

I wonder if there any reason or bug could result in the size of commitlog exceed commitlog_total_space_in_mb ? In our other production environment with the same configurations, Cassandra cluster works well.