[SPARK-16007] Empty DataFrame created with spark.read.csv() does not respect user specified schema

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 2.1.0
• Component/s: SQL
• Target Version/s: 2.1.0
• Assignee: Tathagata Das
• Reporter: Tathagata Das
• Created: 17/Jun/16 02:26
• Updated: 01/Nov/16 21:09
• Resolved: 01/Nov/16 21:09

Description

spark.schema(someSchema).csv().schema != someSchema
The schema of the empty DF created with csv() has no fields.
This problem will occur for json, text, parquet, orc.

Issue Links

• is related to: SPARK-15982 Harmonize the behavior of DataFrameReader.text/csv/json/parquet/orc
• links to: [Github] Pull Request #13727 (tdas)