[CASSANDRA-16072] Reduce thread contention in CommitLogSegment and HintsBuffer by rewriting CAS loops to atomic adds

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.11.12, 4.0.1, 4.1-alpha1, 4.1
• Component/s: Consistency/Hints, Local/Commit Log
• Change Category: Performance
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/f575eea034def1103c5403b11fc8fe58b60d3ebe

Test and Documentation Plan

ci-cassandra.a.o

• Assignee: Michael Semb Wever
• Reporter: Michael Semb Wever

Authors

Michael Semb Wever

Reviewers

Benjamin Lerer, Berenguer Blasi

• Created: 24/Aug/20 10:00
• Updated: 27/May/22 19:25
• Resolved: 01/Aug/21 13:56

Description

Follow up to CASSANDRA-15922

Both CommitLogSegment and HintsBuffer use AtomicIntegers for the current offset when allocating. Like in CASSANDRA-15922 the loops on .compareAndSet(..) can be replaced with atomic adds using the . getAndAdd(..) method.

In highly contended environments the CAS failures can be high, starving writes in a running Cassandra node. On the same cluster CASSANDRA-15922 was found, after CASSANDRA-15922's fix was deployed, there was still problems around commit log flushing and hints. No flamegraph was collected that demonstrated the thread contention as clearly as was found in CASSANDRA-15922, but the performance fix proposed here hopefully is obvious enough.

Issue Links

• relates to: CASSANDRA-15922 High CAS failures in NativeAllocator.Region.allocate(..)