Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
1.3.1
-
None
-
None
Description
When spark sql shell is launched and we point it to a folder containing a large number of parquet files, the sqlContext.parquetFile() command takes a very long time to load the tables.
Attachments
Issue Links
- duplicates
-
SPARK-8125 Accelerate ParquetRelation2 metadata discovery
- Resolved
- is duplicated by
-
SPARK-8838 Add config to enable/disable merging part-files when merging parquet schema
- Resolved