[SPARK-19129] alter table table_name drop partition with a empty string will drop the whole table

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.1.0
• Fix Version/s: 2.1.1, 2.2.0
• Component/s: SQL
• Labels: correctness
• Assignee: Xiao Li
• Reporter: lichenglin
• Created: 09/Jan/17 02:22
• Updated: 17/Jan/17 18:03
• Resolved: 17/Jan/17 18:03

Description

    val spark = SparkSession
      .builder
      .appName("PartitionDropTest")
      .master("local[2]").enableHiveSupport()
      .getOrCreate()
    val sentenceData = spark.createDataFrame(Seq(
      (0, "a"),
      (1, "b"),
      (2, "c")))
      .toDF("id", "name")
    spark.sql("drop table if exists licllocal.partition_table")
    sentenceData.write.mode(SaveMode.Overwrite).partitionBy("id").saveAsTable("licllocal.partition_table")
    spark.sql("alter table licllocal.partition_table drop partition(id='')")
    spark.table("licllocal.partition_table").show()

the result is

|name| id|
+----+---+
+----+---+

Maybe the partition match have something wrong when the partition value is set to empty string

Issue Links

• links to: [Github] Pull Request #16583 (gatorsmile)