[CASSANDRA-19223] [Analytics] Column type mapping error for timestamp type during bulk writes

• Type: Bug
• Status: Resolved
• Priority: Normal
• Resolution: Fixed
• Fix Version/s: NA
• Component/s: Analytics Library
• Bug Category: Correctness - Consistency
• Severity: Normal
• Complexity: Normal
• Discovered By: User Report
• Platform: All
• Since Version: NA

Source Control Link

https://github.com/apache/cassandra-analytics/commit/e82fceaecfe5ea04ac3ddff92be5a6a41456333c

Test and Documentation Plan

Added new integration test to validate the fix

• Assignee: Francisco Guerrero
• Reporter: Francisco Guerrero

Authors

Francisco Guerrero

Reviewers

Yifan Cai

• Created: 21/Dec/23 19:41
• Updated: 10/Jan/24 22:58
• Resolved: 10/Jan/24 22:57

Description

When doing bulk reads with the analytics library, a user can specify the last modified column as an option. Bulk reader will add a column with the last modified column to the data frame. If a user wants to use the bulk-read data frame to persist data, and using the WriterOptions.TIMESTAMP feature from the last modified column from the bulk-read data frame, the bulk write will fail with a data type mapping error.

Caused by: java.lang.RuntimeException: Unsupported conversion for LONG from java.sql.Timestamp
	at org.apache.cassandra.spark.bulkwriter.SqlToCqlTypeConverter$LongConverter.convertInternal(SqlToCqlTypeConverter.java:245)
	at org.apache.cassandra.spark.bulkwriter.SqlToCqlTypeConverter$LongConverter.convertInternal(SqlToCqlTypeConverter.java:231)
	at org.apache.cassandra.spark.bulkwriter.SqlToCqlTypeConverter$Converter.convert(SqlToCqlTypeConverter.java:203)
	at org.apache.cassandra.spark.bulkwriter.SqlToCqlTypeConverter$NullableConverter.convert(SqlToCqlTypeConverter.java:212)
	at org.apache.cassandra.spark.bulkwriter.TableSchema.normalize(TableSchema.java:91)
	at org.apache.spark.api.java.JavaPairRDD$.$anonfun$toScalaFunction$1(JavaPairRDD.scala:1070)
	at scala.collection.Iterator$$anon$10.next(Iterator.scala:461)

Issue Links

• links to: GitHub Pull Request #28