[CASSANDRA-16217] Minimal 4.0 COMPACT STORAGE backport

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 4.0-beta4, 4.0
• Component/s: Legacy/CQL
• Bug Category: Correctness - API / Semantic Implementation
• Severity: Critical
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: 4.0-beta1

Source Control Link

https://github.com/apache/cassandra/commit/8ffec0f02cf73c3d3a8c01aa2d856647a5620a21

Test and Documentation Plan

Tests included; will be additionally tested wit Harry

• Assignee: Alex Petrov
• Reporter: Alex Petrov

Authors

Alex Petrov

Reviewers

Alex Petrov, Marcus Eriksson

• Created: 20/Oct/20 16:54
• Updated: 09/Jan/23 15:36
• Resolved: 08/Dec/20 07:25

Description

There are several behavioural changes related to compact storage, and these differences are larger than most of us have anticipated: we first thought there’ll be that “appearing column”, but there’s implicit nulls in clusterings thing, and row vs column deletion.

Some of the recent issues on the subject are: CASSANDRA-16048, which allows to ignore these differences. The other one was trying to improve user experience of anyone still using compact storage: CASSANDRA-15811.

Easily reproducible differernces are:

(1) hidden columns show up, which breaks SELECT * queries
(2) DELETE v and UPDATE v WITH TTL would result into row removals in non-dense compact tables (CASSANDRA-16069)
(3) INSERT allows skipping clusterings, which are filled with nulls by default.

Some of these are tricky to support, as 15811 has shown. Anyone on OSS side who might want to upgrade to 4.0 while still using compact storage might be affected by being forced into one of these behaviours.

Possible solutions are to document these behaviours, or to bring back a minimal set of COMPACT STORAGE to keep supporting these.

It looks like it is possible to leave some of the functionality related to DENSE flag and allow it to be present in 4.0, but only for these three (and potential related, however not direrclty visible) cases.

e.dimitrova since you were working on removal on compact storage, wanted to reassure that this is not a revert of your patch. On contrary: your patch was instrumental in identifying the right places.

cc slebresne aleksey benedict marcuse

patch

ci

Issue Links

• relates to: CASSANDRA-16226 COMPACT STORAGE SSTables created before 3.0 are not correctly skipped by timestamp due to missing primary key liveness info
• relates to: CASSANDRA-16361 DROP COMPACT STORAGE should invalidate prepared statements still using CompactTableMetadata
• relates to: CASSANDRA-17507 IllegalArgumentException in query code path during 3.11.12 => 4.0.3 rolling upgrade