[CASSANDRA-10855] Use Caffeine (W-TinyLFU) for on-heap caches

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0-alpha1, 4.0
• Labels: performance
• Assignee: Ben Manes
• Reporter: Ben Manes

Authors

Ben Manes

Reviewers

Robert Stupp

• Created: 14/Dec/15 07:01
• Updated: 25/Jul/22 10:53
• Resolved: 01/Jan/17 10:19

Description

Cassandra currently uses ConcurrentLinkedHashMap for performance critical caches (key, counter) and Guava's cache for non-critical (auth, metrics, security). All of these usages have been replaced by Caffeine, written by the author of the previously mentioned libraries.

The primary incentive is to switch from LRU policy to W-TinyLFU, which provides near optimal hit rates. It performs particularly well in database and search traces, is scan resistant, and as adds a very small time/space overhead to LRU.

Secondarily, Guava's caches never obtained similar performance to CLHM due to some optimizations not being ported over. This change results in faster reads and not creating garbage as a side-effect.

Issue Links

• breaks: CASSANDRA-13113 test failure in auth_test.TestAuth.system_auth_ks_is_alterable_test
• breaks: CASSANDRA-13367 CASSANDRA-10855 breaks authentication: throws server error instead of bad credentials on cache load failure
• causes: CASSANDRA-15177 Reloading of auth caches happens on the calling thread
• is duplicated by: CASSANDRA-9911 Remove ConcurrentLinkedHashCache
• is related to: CASSANDRA-11452 Cache implementation using LIRS eviction for in-process page cache