[SPARK-6315] SparkSQL 1.3.0 (RC3) fails to read parquet file generated by 1.1.1

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.1, 1.4.0
• Component/s: SQL
• Target Version/s: 1.3.1
• Assignee: Cheng Lian
• Reporter: Michael Armbrust
• Created: 13/Mar/15 06:40
• Updated: 21/Mar/15 03:20
• Resolved: 21/Mar/15 03:20

Description

Parquet files generated by Spark 1.1 have a deprecated representation of the schema. In Spark 1.3 we fail to read these files through the new Parquet code path. We should continue to read these files until we formally deprecate this representation.

As a workaround:

SET spark.sql.parquet.useDataSourceApi=false

Issue Links

• links to: [Github] Pull Request #5034 (liancheng)