[CASSANDRA-18116] Denylist can load perpetually and too frequently if it fails persistently

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.1.x
• Component/s: Feature/Rate Limiting
• Bug Category: Degradation - Other Exception
• Severity: Normal
• Complexity: Low Hanging Fruit
• Discovered By: User Report
• Platform: All
• Since Version: 4.1.1

Source Control Link

https://github.com/apache/cassandra/commit/073f7c36fa20e8d9410f306e57e7c7734ce74d1e

Test and Documentation Plan

Existing tests (exploring if I can add one more to produce this case and show how reads trigger a load of the denylist)

• Assignee: Jordan West
• Reporter: Jordan West

Authors

Jordan West

Reviewers

Josh McKenzie

• Created: 13/Dec/22 20:19
• Updated: 07/Jan/23 22:01
• Resolved: 07/Jan/23 22:01

Description

When the denylist fails its initial load it can return a value of null for a key in the cache. The BoundedLoadingCache implementation that is used treats null as missing and will end up trying to load the key on reads. Besides the performance impact this can also lead to significant log spam leading to disk or CPU usage issues. 

Issue Links

• relates to: CASSANDRA-12106 Add ability to blocklist / denylist a CQL partition so all requests are ignored