Details
-
Bug
-
Status: Resolved
-
Normal
-
Resolution: Fixed
-
3.0.26, 3.11.12, 4.0.1, 4.1-alpha1, 4.1
-
None
-
Correctness - Recoverable Corruption / Loss
-
Normal
-
Normal
-
User Report
-
All
-
None
-
Description
Secondary indexes on primary key columns can miss some writes. For example, an update after a deletion won't create an index entry:
CREATE TABLE t (pk int, ck int, v int, PRIMARY KEY (pk, ck)); CREATE INDEX ON t(ck); INSERT INTO t(pk, ck, v) VALUES (1, 2, 3); -- creates an index entry (right) DELETE FROM t WHERE pk = 1 AND ck = 2; -- deletes the previous index entry (right) UPDATE t SET v = 3 WHERE pk = 1 AND ck = 2; -- doesn't create a new index entry (wrong) SELECT * FROM t WHERE ck = 2; -- doesn't find the row (wrong)
This happens because the update uses the LivenssInfo of the previously deleted row (see here). The same happens when updating an expired row:
CREATE TABLE t (pk int, ck int, v int, PRIMARY KEY (pk, ck)); CREATE INDEX ON t(ck); UPDATE t USING TTL 1 SET v = 3 WHERE pk = 1 AND ck = 2; -- creates a non-expiring index entry (right) -- wait for the expiration of the above row SELECT * FROM t WHERE ck = 2; -- deletes the index entry (right) UPDATE t SET v = 3 WHERE pk = 1 AND ck = 2; -- doesn't create an index entry (wrong) SELECT * FROM t WHERE ck = 2; -- doesn't find the row (wrong)
I think that the fix for this is just using the getPrimaryKeyIndexLiveness in updateRow, as it's used in insertRow.
Another related problem is that getPrimaryKeyIndexLiveness uses the most recent TTL in the columns contained on the indexed row fragment as the TTL of the index entry, producing an expiring index entry that ignores the columns without TTL that are already present in flushed sstables. So we can find this other error when setting a TTL over flushed indexed data:
CREATE TABLE t(k1 int, k2 int, v int, PRIMARY KEY ((k1, k2))); CREATE INDEX idx ON t(k1); INSERT INTO t (k1, k2, v) VALUES (1, 2, 3); -- flush UPDATE t USING TTL 1 SET v=0 WHERE k1=1 AND k2=2; -- creates an index entry with TTL (wrong) -- wait for TTL expiration SELECT TTL(v) FROM t WHERE k1=1; -- doesn't find the row (wrong)
The straightforward fix is just ignoring the TTL of the columns for indexes on primary key components, so we don't produce expiring index entries in that case. The index entries will be eventually deleted during index reads, when we are sure that they are not pointing to any live data.
