Details
-
Improvement
-
Status: Closed
-
Minor
-
Resolution: Fixed
-
1.8.0
Description
Currently, records which have null fields are written to their corresponding columns in Cassandra as null. Writing null is basically a 'delete' for Cassandra, it's useful if nulls should correspond to deletes in the data model, but nulls can also indicate a missing data or partial column update. In that case, we end up overwriting columns of existing record on Cassandra with nulls.
I believe it's already possible to ignore null values for POJO's with mapper options, as documented here:
But this is not possible when using scala tuples or case classes. Perhaps with a Cassandra sink configuration flag, null values can be unset using below option for tuples and case classes.
Here is the equivalent configuration in spark-cassandra-connector;
Attachments
Issue Links
- links to