[CASSANDRA-16883] Weak visibility guarantees of Accumulator can lead to failure to recognize digest mismatches

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.26, 3.11.12
• Component/s: Consistency/Coordination
• Bug Category: Correctness - Consistency
• Severity: Critical
• Complexity: Normal
• Discovered By: Fuzz Test
• Platform: All
• Since Version: 3.0.0

Source Control Link

https://github.com/apache/cassandra/commit/f9d41ff83655ead37ac6083d7ee43f2c35a346da

Test and Documentation Plan

new unit test in DigestResolverTest covers the changes here

• Assignee: Caleb Rackliffe
• Reporter: Caleb Rackliffe

Authors

Caleb Rackliffe

Reviewers

Jon Meredith

• Created: 25/Aug/21 17:51
• Updated: 01/Sep/21 09:35
• Resolved: 31/Aug/21 21:43

Description

The context for this problem is largely the same as CASSANDRA-16807. The difference is that for 4.0+, CASSANDRA-16097 added an assertion to DigestResolver#responseMatch() that ensures the responses snapshot has at least one visible element (although of course only one element trivially cannot generate a mismatch and short-circuits immediately). In 3.0 and 3.11, this assertion does not exist, and when the underlying problem occurs (i.e. zero responses are visible on Accumulator when there should be 2), we can silently avoid the digest matching entirely. This seems like it would make it both impossible to do a potentially necessary full data read to resolve the correct response and prevent repair.

The fix here should be similar to the one in CASSANDRA-16807, although there might be some test infrastructure that needs porting in order to make that work.

Issue Links

• is related to: CASSANDRA-16807 Weak visibility guarantees of Accumulator lead to failed assertions during digest comparison