[SPARK-13543] Support for specifying compression codec for Parquet/ORC via option()

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.0
• Component/s: SQL
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 29/Feb/16 03:29
• Updated: 12/Dec/22 18:11
• Resolved: 03/Mar/16 18:31

Description

Likewise, SPARK-12871, SPARK-12872 and SPARK-13503, the compression codec can be set via option() for Parquet and ORC rather than manually setting them to Hadoop configuration.

Issue Links

• is duplicated by: SPARK-7051 Support Compression write for Parquet
• relates to: SPARK-12307 ParquetFormat options should be exposed through the DataFrameReader/Writer options API
• links to: [Github] Pull Request #11464 (HyukjinKwon)