[CASSANDRA-16675] Preserve Query Performance with ClusteringIndexNamesFilter After Running DROP COMPACT STORAGE

• Type: Improvement
• Status: Open
• Priority: Normal
• Resolution: Unresolved
• Fix Version/s: 4.0.x
• Component/s: Legacy/Local Write-Read Paths
• Change Category: Performance
• Complexity: Normal
• Platform: All
• Assignee: Caleb Rackliffe
• Reporter: Caleb Rackliffe

Authors

Caleb Rackliffe

• Created: 18/May/21 18:00
• Updated: 20/Jul/21 20:07

Description

Before the completion of CASSANDRA-16226, upgrading a cluster from 2.1 to 3.0 with compact tables could cause a significant regression in the latency of reads using ClusteringIndexNamesFilter. The details are described in that Jira, but in short, 3.0+ did not skip SSTables it should have during reads, because it thought (wrongly) there might be primary key liveness information in SSTables for compact tables.

CASSANDRA-16226 addressed this behavior for still-compact tables, and also maintained it after DROP COMPACT STORAGE was run. However, it also allowed tables that were never compact to drop rows from query results if they contained no live non-key columns, which is only a normal behavior for compact tables. This is addressed in CASSANDRA-16671 by reverting the bits of the logic from CASSANDRA-16226 that deal with formerly compact tables where DROP COMPACT STORAGE has been run, in the interest of unblocking the 4.0 release and making sure strictly compact and strictly non-compact tables are queried properly and construct properly formed results.

This goal of this issue is to safely restore the performance of formerly compact tables, which necessarily contain ambiguous primary key liveness info. Roughly, the idea is that we record in a system table (and pull into TableMetadata) the time when DROP COMPACT STORAGE is executed. If a time exists for a table, we can treat it as being formerly compact, and ignore primary key liveness info for determining row completeness in SinglePartitionReadCommand#isComplete(). Otherwise, the normal rules for never-compact tables will apply, avoiding any regression in the scenario described by CASSANDRA-16671.

This would obviously not be helpful in the case where a user has already dropped compact storage, but it may logically be the best we can do, given we cannot correctly reconstruct liveness info for SSTables created while a table was compact (i.e. there is no way to tell INSERT and UPDATE apart for those). Especially if CASSANDRA-16671 moves in the direction of disabling DROP COMPACT STORAGE by default, I would also propose that we do this only for 4.0+.

Issue Links

• is related to: CASSANDRA-16226 COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
• is related to: CASSANDRA-16671 Cassandra can return no row when the row columns have been deleted.