[CASSANDRA-1930] db.Table flusherLock write lock fairness policy is sub-optimal

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 0.7.1
• Assignee: Kelvin Kakugawa
• Reporter: Kelvin Kakugawa

Authors

Kelvin Kakugawa

Reviewers

Jonathan Ellis

• Created: 04/Jan/11 01:18
• Updated: 16/Apr/19 09:33
• Resolved: 06/Jan/11 22:58

Description

scenario:

1) high write throughput cluster
2) external services adding material cpu contention

symptoms:

The row mutation stage falls very behind.

cause:

ReentrantReadWriteLock's fair policy causes write lock acquisition / release to require a material amount of CPU time.

summary:

When there are other services contending for the CPU, the RRW lock's fair policy causes write lock acquisition / release to take enough time to eventually put threads waiting for read lock acquisition very behind. We repro'd this scenario by reducing the scope of the write lock to: 1 boolean check, 1 boolean assignment, and 1 db.Memtable instantiation (itself, just: 2 variable assignments) w/ the same performance. Modifying the fairness policy to be the default policy allowed the row mutation stage to keep up.

Issue Links

• is related to: CASSANDRA-1954 Double-check or replace RRW memtable lock