[SPARK-17732] ALTER TABLE DROP PARTITION should support comparators

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.0.0
• Component/s: SQL
• Target Version/s: 2.2.0
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 29/Sep/16 20:11
• Updated: 06/Sep/18 13:09
• Resolved: 26/Mar/17 19:16

Description

This issue aims to support `comparators`, e.g. '<', '<=', '>', '>=', again in Apache Spark 2.0 for backward compatibility.

Spark 1.6.2

scala> sql("CREATE TABLE sales(id INT) PARTITIONED BY (country STRING, quarter STRING)")
res0: org.apache.spark.sql.DataFrame = [result: string]

scala> sql("ALTER TABLE sales DROP PARTITION (country < 'KR')")
res1: org.apache.spark.sql.DataFrame = [result: string]

Spark 2.0

scala> sql("CREATE TABLE sales(id INT) PARTITIONED BY (country STRING, quarter STRING)")
res0: org.apache.spark.sql.DataFrame = []

scala> sql("ALTER TABLE sales DROP PARTITION (country < 'KR')")
org.apache.spark.sql.catalyst.parser.ParseException:
mismatched input '<' expecting {')', ','}(line 1, pos 42)

Issue Links

• duplicates: SPARK-14922 Alter Table Drop Partition Using Predicate-based Partition Spec
• is duplicated by: SPARK-18515 AlterTableDropPartitions fails for non-string columns
• links to: [Github] Pull Request #15302 (dongjoon-hyun)
• links to: [Github] Pull Request #15704 (dongjoon-hyun)
• links to: [Github] Pull Request #15948 (hvanhovell)
• links to: [Github] Pull Request #15987 (dongjoon-hyun)
• links to: [Github] Pull Request #16036 (cloud-fan)
• links to: [Github] Pull Request #19691 (DazhuangSu)
• links to: [Github] Pull Request #20999 (mgaido91)