[SPARK-29908] Support partitioning for DataSource V2 tables in DataFrameWriter.save

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Duplicate
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Burak Yavuz
• Created: 15/Nov/19 04:09
• Updated: 12/Dec/22 18:10
• Resolved: 19/Feb/20 17:00

Description

Currently, any data source that that upgrades to DataSource V2 loses the partition transform information when using DataFrameWriter.save. The main reason is the lack of an API for "creating" a table with partitioning and schema information for V2 tables without a catalog.

Issue Links

• is duplicated by: SPARK-29219 DataSourceV2: Support all SaveModes in DataFrameWriter.save
• links to: PR-25822
• links to: PR-25833