[CASSANDRA-19048] Audit table properties passed through Analytics CqlUtils

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: NA
• Component/s: Analytics Library
• Bug Category: Correctness
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: 5.0-alpha2

Source Control Link

https://github.com/apache/cassandra-analytics/commit/c73c76498b0c2b36705025de6b0b2a7bb38e758b

Test and Documentation Plan

ci

• Assignee: Doug Rohrer
• Reporter: Doug Rohrer

Authors

Doug Rohrer

Reviewers

Dinesh Joshi, Francisco Guerrero, Yifan Cai

• Created: 21/Nov/23 20:27
• Updated: 30/Nov/23 15:26
• Resolved: 30/Nov/23 05:37

Description

In mixed version clusters, table properties (`WITH` clauses in the table definition) have historically been moved or renamed without much thought to backward-compatibility of external tools. This has caused breaking changes to the Analytics library in the past. Therefore, the Analytics library removes most table properties from the table schema it uses in order to avoid these issues. However, it is currently too aggressive in this removal process.  For example, the `default_time_to_live` parameter is currently removed from the table options, but the Bulk Writer should use this to set the appropriate TTL when the user doesn’t specify it in the bulk job.

Similarly, compression options passed to the Bulk Writer should be honored so that the appropriate compression algorithm is used when generating sstables.

The allowlist of options needs to be audited and improved to add additional options where appropriate.

Issue Links

• links to: GitHub Pull Request #21