[SPARK-34203] FileSource table null partition can not be dropped

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.1, 3.1.0, 3.2.0
• Fix Version/s: 3.0.2, 3.1.1, 3.2.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Kent Yao 2
• Created: 22/Jan/21 11:19
• Updated: 08/Feb/21 13:54
• Resolved: 25/Jan/21 22:45

Description

case to reproduce

create table ppp(i string, j string) using parquet partitioned by (j);

INSERT INTO ppp PARTITION (j=null) VALUES ('1');
alter table ppp drop partition(j=null); — success

INSERT OVERWRITE ppp VALUES ('1', null); — error
Error in query: No partition is dropped. One partition spec 'Map(j -> null)' does not exist in table 'ppp' database 'default';

alter table ppp drop partition(j='_HIVE_DEFAULT_PARTITION_'); — then, this is need

Issue Links

• links to: [Github] Pull Request #31322 (MaxGekk)
• links to: [Github] Pull Request #31326 (MaxGekk)
• links to: [Github] Pull Request #31331 (MaxGekk)