[SPARK-8785] Improve Parquet schema merging

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.5.0
• Component/s: SQL
• Assignee: L. C. Hsieh
• Reporter: L. C. Hsieh
• Shepherd: Cheng Lian
• Created: 02/Jul/15 07:30
• Updated: 08/Jul/15 17:10
• Resolved: 08/Jul/15 17:10

Description

Currently, the parquet schema merging (ParquetRelation2.readSchema) may spend much time to merge duplicate schema. We can select only non duplicate schema and merge them later.

Issue Links

• links to: [Github] Pull Request #7182 (viirya)