[SPARK-33904] Recognize `spark_catalog` in `saveAsTable()` and `insertInto()`

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.2.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 24/Dec/20 17:41
• Updated: 30/Dec/20 07:56
• Resolved: 30/Dec/20 07:56

Description

The v1 INSERT INTO command recognizes `spark_catalog` as the default session catalog:

spark-sql> create table spark_catalog.ns.tbl (c int);
spark-sql> insert into spark_catalog.ns.tbl select 0;
spark-sql> select * from spark_catalog.ns.tbl;
0

but the `saveAsTable()` and `insertInto()` methods don't allow to write a table with explicitly specified catalog spark_catalog:

scala> sql("CREATE NAMESPACE spark_catalog.ns")
scala> Seq(0).toDF().write.saveAsTable("spark_catalog.ns.tbl")
org.apache.spark.sql.AnalysisException: Couldn't find a catalog to handle the identifier spark_catalog.ns.tbl.
  at org.apache.spark.sql.DataFrameWriter.saveAsTable(DataFrameWriter.scala:629)
  ... 47 elided
scala> Seq(0).toDF().write.insertInto("spark_catalog.ns.tbl")
org.apache.spark.sql.AnalysisException: Couldn't find a catalog to handle the identifier spark_catalog.ns.tbl.
  at org.apache.spark.sql.DataFrameWriter.insertInto(DataFrameWriter.scala:498)
  ... 47 elided

Issue Links

• links to: [Github] Pull Request #30919 (MaxGekk)
• links to: [Github] Pull Request #30919 (MaxGekk)