[SPARK-33676] Require exact matched partition spec to schema in ADD/DROP PARTITION

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 06/Dec/20 09:13
• Updated: 07/Dec/20 08:15
• Resolved: 07/Dec/20 08:15

Description

The V1 implementation of ALTER TABLE .. ADD/DROP PARTITION fails when the partition spec doesn't exactly match to the partition schema:

ALTER TABLE tab1 ADD PARTITION (A='9')
Partition spec is invalid. The spec (a) must match the partition spec (a, b) defined in table '`dbx`.`tab1`';
org.apache.spark.sql.AnalysisException: Partition spec is invalid. The spec (a) must match the partition spec (a, b) defined in table '`dbx`.`tab1`';
	at org.apache.spark.sql.catalyst.catalog.SessionCatalog.$anonfun$requireExactMatchedPartitionSpec$1(SessionCatalog.scala:1173)
	at org.apache.spark.sql.catalyst.catalog.SessionCatalog.$anonfun$requireExactMatchedPartitionSpec$1$adapted(SessionCatalog.scala:1171)
	at scala.collection.immutable.List.foreach(List.scala:392)

for a table partitioned by "a", "b" but the V2 implementation add the wrong partition silently.

Issue Links

• links to: [Github] Pull Request #30624 (MaxGekk)