[CASSANDRA-18294] die disk failure policy will not kill jvm as documented

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.29, 4.0.9, 4.1.1, 5.0-alpha1, 5.0
• Component/s: Local/Config
• Bug Category: Correctness
• Severity: Low
• Complexity: Low Hanging Fruit
• Discovered By: Code Inspection
• Platform: All
• Since Version: 3.0.22

Source Control Link

https://github.com/apache/cassandra/commit/be574fc34ba9834929f1618ab63dd74446cd2683

Test and Documentation Plan

run CI

• Assignee: Runtian Liu
• Reporter: Runtian Liu

Authors

Runtian Liu

Reviewers

Brandon Williams, Maxwell Guo, Stefan Miklosovic

• Created: 01/Mar/23 23:02
• Updated: 12/Sep/23 13:02
• Resolved: 09/Mar/23 16:07

Description

After Cassandra has successfully starts up with disk_failure_policy die, when encounter a file system error, Cassandra server will only throw exception instead of shut down gossip and client transport and kill JVM. Document: https://cassandra.apache.org/doc/latest/cassandra/configuration/cass_yaml_file.html#disk_failure_policy

 

The reason for this is the default FS error handler is not handing policy die correctly. Instead of shutting down gossip and native transport, it throws an error.

 

The JVMStabilityInspectorTest is passing because the error handler is not set so no exception is thrown.

Issue Links

• is caused by: CASSANDRA-15191 stop_paranoid disk failure policy is ignored on CorruptSSTableException after node is up