[CASSANDRA-19727] [Analytics] Bulk writer fails validation stage when writing to a cluster using RandomPartitioner

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: NA
• Component/s: Analytics Library
• Bug Category: Correctness - Unrecoverable Corruption / Loss
• Severity: Low
• Complexity: Low Hanging Fruit
• Discovered By: User Report
• Platform: All
• Since Version: NA

Source Control Link

https://github.com/apache/cassandra-analytics/commit/b31a451873eb411788a6d94c3cacf881f5c3cb86

Test and Documentation Plan

Added tests for RandomPartitioner.

• Assignee: Francisco Guerrero
• Reporter: Francisco Guerrero

Authors

Francisco Guerrero

Reviewers

Yifan Cai, Yifan Cai

• Created: 26/Jun/24 22:27
• Updated: 27/Jun/24 22:51
• Resolved: 27/Jun/24 22:51

Description

In bulk writer after writing SSTables, written data to SSTables is read back again to perform validations before shipping the files to the Cassandra instances. The logic to validate the SSTables assumes Murmur3Partitioner. This validation fails however when a bulk writer job is running against a cluster using the RandomPartitioner with the following stacktrace:

java.lang.RuntimeException: java.lang.IllegalStateException: Partitioner in ValidationMetadata does not match TableMetaData: org.apache.cassandra.dht.RandomPartitioner vs. org.apache.cassandra.dht.Murmur3Partitioner
	at org.apache.cassandra.spark.utils.Throwing.lambda$function$2(Throwing.java:84)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:197)
	at java.base/java.util.HashMap$KeySpliterator.forEachRemaining(HashMap.java:1707)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at org.apache.cassandra.spark.data.BasicSupplier.openAll(BasicSupplier.java:44)
	at org.apache.cassandra.bridge.CassandraBridgeImplementation.getCompactionScanner(CassandraBridgeImplementation.java:248)
	at org.apache.cassandra.spark.data.DataLayer.openCompactionScanner(DataLayer.java:262)
	at org.apache.cassandra.spark.bulkwriter.SSTableWriter.validateSSTables(SSTableWriter.java:129)
	at org.apache.cassandra.spark.bulkwriter.SSTableWriter.close(SSTableWriter.java:113)
	at org.apache.cassandra.spark.bulkwriter.RecordWriter.finalizeSSTable(RecordWriter.java:224)
	at org.apache.cassandra.spark.bulkwriter.RecordWriter.write(RecordWriter.java:122)
	... 15 more