[CASSANDRA-17812] Rate-limit new client connection setup to avoid overwhelming bcrypt

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.0-alpha1, 5.0
• Component/s: Feature/Encryption
• Change Category: Operability
• Complexity: Normal
• Platform: All

Source Control Link

https://gitbox.apache.org/repos/asf?p=cassandra.git;a=commit;h=09b282d1fdd7d6d62542137003011d144c0227be

Test and Documentation Plan

New unit testing

• Assignee: Josh McKenzie
• Reporter: Josh McKenzie

Authors

Josh McKenzie

Reviewers

Caleb Rackliffe

• Created: 11/Aug/22 17:31
• Updated: 05/Nov/24 16:49
• Resolved: 15/Aug/22 14:33

Description

A flood of reconnects can cause a ton of pain at the bcrypt phase of validating incoming connections. While this shouldn't happen during normal operations, we need a rate limit server side - if there's a bad client out there (version and/or configuration) that misbehaves, a way to cap the pain on a server is quite useful. Right now the only option is to cap the total connections which has other issues that aren't an ideal tradeoff (inability to connect, etc).

Moving authentication requests to a small, separate pool will prevent starvation handling all other requests. If the authExecutor pool backs up it may cause authentication timeouts, but the clients should back off and retry while the rest of the system continues to make progress.

Issue Links

• is related to: CASSANDRA-20057 Backport CASSANDRA-17812: Rate-limit new client connection auth setup to avoid overwhelming bcrypt
• relates to: CASSANDRA-18541 AUTH requests use too much resources