[SPARK-22926] Respect table-level conf compression codec `Compression` in multiple scenarios

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.0
• Component/s: SQL
• Labels: bulk-closed
• Assignee: Xiao Li
• Reporter: Xiao Li
• Created: 30/Dec/17 05:59
• Updated: 08/Oct/19 05:42
• Resolved: 08/Oct/19 05:42

Description

Currently, we ignore table-specific `compression` conf when the Hive serde tables are converted to the data source tables. We also ignore it when users set `compression` in the TBLPROPERTIES instead of OPTIONS, even if the tables are native data source tables.

Issue Links

• is related to: SPARK-22158 convertMetastore should not ignore storage properties
• links to: [Github] Pull Request #20120 (gatorsmile)