Details
-
Bug
-
Status: Resolved
-
Trivial
-
Resolution: Fixed
-
3.5.1
Description
In FileSourceStrategy, the schema of the data excluding partition columns is computed 2 times in a slightly different way:
val dataColumnsWithoutPartitionCols = dataColumns.filterNot(partitionSet.contains)
vs
val readDataColumns = dataColumns .filterNot(partitionColumns.contains)
Attachments
Issue Links
- relates to
-
SPARK-48241 CSV parsing failure with char/varchar type columns
- Resolved
- links to