Description
scala> sql("create table sample.sample stored as SEQUENCEFILE as select 1 as key, 'abc' as value")
res2: org.apache.spark.sql.DataFrame = []
scala> val df = sql("select key, value as value from sample.sample")
df: org.apache.spark.sql.DataFrame = [key: int, value: string]
scala> df.write.mode("append").saveAsTable("sample.sample")
scala> sql("select * from sample.sample").show()
+---+-----+
|key|value|
+---+-----+
| 1| abc|
| 1| abc|
+---+-----+
In Spark 1.6, it works, but Spark 2.0 does not work. The error message from Spark 2.0 is
scala> df.write.mode("append").saveAsTable("sample.sample")
org.apache.spark.sql.AnalysisException: Saving data in MetastoreRelation sample, sample
is not supported.;
So far, we do not plan to support it in Spark 2.0. Spark 1.6 works because it internally uses insertInto. But, if we change it back it will break the semantic of saveAsTable (this method uses by-name resolution instead of using by-position resolution used by insertInto).
Instead, users should use insertInto API. We should correct the error messages. Users can understand how to bypass it before we support it.
Attachments
Issue Links
- is duplicated by
-
SPARK-16789 Can't run saveAsTable with database name
-
- Closed
-
- links to
