When user configures nullValue in CSV data source, in addition to those values, all empty string values are also converted to null.
data:
col1,col2
1,"-"
2,""
spark.read.format("csv").option("nullValue", "-")
We will find a null in both rows.
- is duplicated by
-
SPARK-21768 spark.csv.read Empty String Parsed as NULL when nullValue is Set
-
- Resolved
-
- links to
(2 links to)