[SPARK-6471] Metastore schema should only be a subset of parquet schema to support dropping of columns using replace columns

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.3.0
• Fix Version/s: 1.3.1, 1.4.0
• Component/s: SQL
• Target Version/s: 1.3.1, 1.4.0
• Assignee: Yash Datta
• Reporter: Yash Datta
• Created: 23/Mar/15 20:48
• Updated: 04/Apr/15 15:09
• Resolved: 26/Mar/15 16:14

Description

Currently in the parquet relation 2 implementation, error is thrown in case merged schema is not exactly the same as metastore schema.
But to support cases like deletion of column using replace column command, we can relax the restriction so that even if metastore schema is a subset of merged parquet schema, the query will work.

Issue Links

• is duplicated by: SPARK-6242 Support replace (drop) column for parquet table
• links to: [Github] Pull Request #5141 (saucam)