[CASSANDRA-19334] [Analytics] Upgrade to Cassandra 4.0.12 and remove RowBufferMode and BatchSize options

• Type: Improvement
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: NA
• Component/s: Analytics Library
• Change Category: Performance
• Complexity: Normal
• Platform: All

Source Control Link

https://github.com/apache/cassandra-analytics/commit/e0ae9d7484e242f6af495aac2cb4d8dc121fba89

Test and Documentation Plan

unit; ci

• Assignee: Yifan Cai
• Reporter: Yifan Cai

Authors

Yifan Cai

Reviewers

Francisco Guerrero

• Created: 27/Jan/24 19:53
• Updated: 31/Jan/24 18:54
• Resolved: 31/Jan/24 18:54

Description

In cassandra-all:4.0.12, improvements were made for the CQLSSTableWriter. The sorted writer now can produce size-capped SSTables. It replaces the need for the unsorted sstable writer, which has to buffer and sort data on flushing. The dataset to write in the spark application is already sorted. By avoiding using the unsorted writer, it prevents wasting CPU time on sorting the sorted data. Since the sorted sstable writer does not need to buffer data, its size estimation is more accurate than the unsorted one, meaning the produced sstables files are closer to the expectation.

By removing the unsorted sstable writer, it no longer requires the RowBufferMode option.
By supporting size-capping in sorted writer, it no longer requires the BatchSize option.

Issue Links

• links to: GitHub Pull Request #36