Description
When casting given string datum to specified type, CSV should return null for nullable types if datum == options.nullValue.
However, for certain data types like Boolean, TimestampType, DateType, CSV in 2.0 does not return null for some "empty" datum. This is a regression comparing to 1.6.
Attachments
Issue Links
- is duplicated by
-
SPARK-16903 nullValue in first field is not respected by CSV source when read
- Resolved
-
SPARK-17039 cannot read null dates from csv file
- Resolved
-
SPARK-17290 Spark CSVInferSchema does not always respect nullValue settings
- Resolved
-
SPARK-15144 option nullValue for CSV data source not working for several types.
- Resolved
- is related to
-
SPARK-16460 Spark 2.0 CSV ignores NULL value in Date format
- Resolved
- relates to
-
SPARK-14143 Options for parsing NaNs, Infinity and nulls for numeric types
- Resolved
- links to