[CASSANDRA-10726] Read repair inserts should not be blocking

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0-alpha1, 4.0
• Component/s: Legacy/Coordination
• Assignee: Blake Eggleston
• Reporter: Richard Low

Authors

Blake Eggleston

Reviewers

Alex Petrov, Marcus Eriksson

• Created: 17/Nov/15 22:24
• Updated: 16/Mar/22 11:04
• Resolved: 21/Aug/18 18:05

Description

Today, if there’s a digest mismatch in a foreground read repair, the insert to update out of date replicas is blocking. This means, if it fails, the read fails with a timeout. If a node is dropping writes (maybe it is overloaded or the mutation stage is backed up for some other reason), all reads to a replica set could fail. Further, replicas dropping writes get more out of sync so will require more read repair.

The comment on the code for why the writes are blocking is:

// wait for the repair writes to be acknowledged, to minimize impact on any replica that's
// behind on writes in case the out-of-sync row is read multiple times in quick succession

but the bad side effect is that reads timeout. Either the writes should not be blocking or we should return success for the read even if the write times out.

Issue Links

• breaks: CASSANDRA-14740 BlockingReadRepair does not maintain monotonicity during range movements
• is duplicated by: CASSANDRA-14480 Digest mismatch requires all replicas to be responsive
• is related to: CASSANDRA-14635 Support table level configuration of monotonic reads
• relates to: CASSANDRA-13863 Speculative retry causes read repair even if read_repair_chance is 0.0.
• links to: Dtest
• links to: GitHub Pull Request #94