[SPARK-16610] When writing ORC files, orc.compress should not be overridden if users do not set "compression" in the options

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: SQL
• Target Version/s: 2.0.1, 2.1.0
• Assignee: Hyukjin Kwon
• Reporter: Yin Huai
• Created: 18/Jul/16 18:22
• Updated: 12/Dec/22 18:11
• Resolved: 09/Aug/16 02:25

Description

For ORC source, Spark SQL has a writer option compression, which is used to set the codec and its value will be also set to orc.compress (the orc conf used for codec). However, if a user only set orc.compress in the writer option, we should not use the default value of "compression" (snappy) as the codec. Instead, we should respect the value of orc.compress.

Issue Links

• links to: [Github] Pull Request #14518 (HyukjinKwon)