[SPARK-30510] Publicly document options under spark.sql.*

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Documentation, SQL
• Labels: release-notes
• Assignee: Hyukjin Kwon
• Reporter: Nicholas Chammas
• Created: 14/Jan/20 18:37
• Updated: 12/Dec/22 18:10
• Resolved: 09/Feb/20 10:22

Description

SPARK-20236 added a new option, spark.sql.sources.partitionOverwriteMode, but it doesn't appear to be documented in the expected place. In fact, none of the options under spark.sql.* that are intended for users are documented on spark.apache.org/docs.

We should add a new documentation page for these options.

Issue Links

• relates to: SPARK-20236 Overwrite a partitioned data source table should only overwrite related partitions
• links to: GitHub Pull Request #27459