[SPARK-23039] Fix the bug in alter table set location.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Won't Fix
• Affects Version/s: 2.2.1
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Bo Xu
• Created: 11/Jan/18 07:45
• Updated: 24/Sep/18 03:20
• Resolved: 24/Sep/18 03:20

Description

TOBO work: Fix the bug in alter table set location.
org.apache.spark.sql.execution.command.DDLSuite#testSetLocation

    // TODO(gatorsmile): fix the bug in alter table set location.
   //    if (isUsingHiveMetastore) {
    //    assert(storageFormat.properties.get("path") === expected)
    //   }

Analysis:

because user add locationUri and erase path by

 newPath = None

in org.apache.spark.sql.hive.HiveExternalCatalog#restoreDataSourceTable:

val storageWithLocation = {
      val tableLocation = getLocationFromStorageProps(table)
      // We pass None as `newPath` here, to remove the path option in storage properties.
      updateLocationInStorageProps(table, newPath = None).copy(
        locationUri = tableLocation.map(CatalogUtils.stringToURI(_)))
    }

=>
newPath = None

Issue Links

• duplicates: SPARK-23057 SET LOCATION should change the path of partition in table
• links to: [Github] Pull Request #20260 (xubo245)