Details
-
Improvement
-
Status: Resolved
-
Trivial
-
Resolution: Fixed
-
2.3.0
-
None
Description
As a follow-up to SPARK-20297 (and SPARK-10400) in which spark.sql.parquet.writeLegacyFormat property was recommended for Impala and Hive, Spark SQL docs for Parquet Files should have it documented.
p.s. It was asked about in Why can't Impala read parquet files after Spark SQL's write? on StackOverflow today.
p.s. It's also covered in holden.karau@gmail.com's "High Performance Spark: Best Practices for Scaling and Optimizing Apache Spark" book (in Table 3-10. Parquet data source options) that gives the option some wider publicity.