[CASSANDRA-18490] Checksum per-SSTable and per-column SAI components after streaming

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: 5.x
• Component/s: Feature/2i Index
• Epic Link: CEP-7 Storage Attached Index (SAI)
• Change Category: Operability
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra/commit/9057ff3e6dad89b0197d83f404a9b0fca544e0f7

Test and Documentation Plan

• Added checksum validation unit tests
• Verified logs of streaming tests manually and confirmed checksum validation is being called

• Assignee: Piotr Kolaczkowski
• Reporter: Mike Adamson

Authors

Piotr Kolaczkowski

Reviewers

Andres de la Peña, Caleb Rackliffe

• Created: 02/May/23 16:55
• Updated: 11/Jul/23 22:02
• Resolved: 11/Jul/23 19:19

Description

The SAI code currently does not checksum validate per-column index data files at any point. It does checksum validate per-sstable components after a full rebuild and it checksum validates the per-column metadata on opening.

We should checksum validate all index components on startup, full rebuild and streaming.

EDIT: The changes we ultimately made here were:

• Fix checksum calculation in IncrementalChecksumSequentialWriter
• Checksum per-SSTable and per-column components after streaming
• Avoid validating indexes when full rebuild is requested

Issue Links

• relates to: CASSANDRA-18653 Avoid unnecessary index builds and validations