[SPARK-11787] Speed up parquet reader for flat schemas

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: SQL
• Assignee: Nong Li
• Reporter: Nong Li
• Created: 17/Nov/15 20:19
• Updated: 19/Nov/15 21:39
• Resolved: 19/Nov/15 02:39

Description

Measuring the performance of running some of the TPCDS and anecdotally, parquet scan and record reconstruction performance is a bottleneck.

For simple schemas, we can do better using the lower level parquet-mr APIs.

Issue Links

• links to: [Github] Pull Request #9774 (nongli)
• links to: [Github] Pull Request #9801 (nongli)
• links to: [Github] Pull Request #9845 (nongli)