[CASSANDRA-8272] 2ndary indexes can return stale data

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 3.0.21, 3.11.7, 4.0-beta1, 4.0
• Component/s: Feature/2i Index
• Labels: pull-request-available
• Bug Category: Correctness - Transient Incorrect Response
• Severity: Normal
• Since Version: 0.3

Source Control Link

https://github.com/apache/cassandra/commit/dd255ffa07d0263521a1ca863fc2192db19bc04c

• Assignee: Andres de la Peña
• Reporter: Sylvain Lebresne

Authors

Andres de la Peña

Reviewers

Benjamin Lerer, Caleb Rackliffe, Zhao Yang

• Created: 06/Nov/14 21:22
• Updated: 16/Nov/23 20:44
• Resolved: 05/Jun/20 12:27

Description

When replica return 2ndary index results, it's possible for a single replica to return a stale result and that result will be sent back to the user, potentially failing the CL contract.

For instance, consider 3 replicas A, B and C, and the following situation:

CREATE TABLE test (k int PRIMARY KEY, v text);
CREATE INDEX ON test(v);
INSERT INTO test(k, v) VALUES (0, 'foo');

with every replica up to date. Now, suppose that the following queries are done at QUORUM:

UPDATE test SET v = 'bar' WHERE k = 0;
SELECT * FROM test WHERE v = 'foo';

then, if A and B acknowledge the insert but C respond to the read before having applied the insert, then the now stale result will be returned (since C will return it and A or B will return nothing).

A potential solution would be that when we read a tombstone in the index (and provided we make the index inherit the gcGrace of it's parent CF), instead of skipping that tombstone, we'd insert in the result a corresponding range tombstone.

Issue Links

• causes: CASSANDRA-16311 Extend the exclusion of replica filtering protection to other indices instead of just SASI
• is depended upon by: CASSANDRA-13423 Secondary indexes can return stale data for deleted rows in 2.x
• is duplicated by: CASSANDRA-8273 Allow filtering queries can return stale data
• is related to: CASSANDRA-8273 Allow filtering queries can return stale data
• relates to: CASSANDRA-19007 Queries with multi-column replica-side filtering can miss rows
• relates to: CASSANDRA-19018 An SAI-specific mechanism to ensure consistency isn't violated for multi-column (i.e. AND) queries at CL > ONE
• relates to: CASSANDRA-15907 Operational Improvements & Hardening for Replica Filtering Protection
• relates to: CASSANDRA-15910 Add multi-partition key commands
• links to: 3.0 PR
• links to: GitHub Pull Request #59
• links to: GitHub Pull Request #486
• links to: GitHub Pull Request #493