Description
The following Spark shell snippet reproduces this bug:
sqlContext range 10 registerTempTable "x" // The problematic DDL statement: sqlContext sql "CREATE TEMPORARY TABLE y USING PARQUET AS SELECT * FROM x" sqlContext.tables().show()
It shows the following result:
+---------+-----------+ |tableName|isTemporary| +---------+-----------+ | y| false| | x| true| +---------+-----------+
Note that y is NOT temporary although it's created using CREATE TEMPORARY TABLE ....
Explain shows that the physical plan node is CreateTableUsingAsSelect rather than CreateTempTableUsingAsSelect.
== Parsed Logical Plan == 'CreateTableUsingAsSelect `y`, PARQUET, true, [Ljava.lang.String;@4d001a14, None, Overwrite, Map() +- 'Project [*] +- 'UnresolvedRelation `x`, None == Analyzed Logical Plan == CreateTableUsingAsSelect `y`, PARQUET, true, [Ljava.lang.String;@4d001a14, None, Overwrite, Map() +- Project [id#0L] +- SubqueryAlias x +- Range 0, 10, 1, 1, [id#0L] == Optimized Logical Plan == CreateTableUsingAsSelect `y`, PARQUET, true, [Ljava.lang.String;@4d001a14, None, Overwrite, Map() +- Range 0, 10, 1, 1, [id#0L] == Physical Plan == ExecutedCommand CreateMetastoreDataSourceAsSelect `y`, PARQUET, [Ljava.lang.String;@4d001a14, None, Overwrite, Map(), Range 0, 10, 1, 1, [id#0L]|
Attachments
Issue Links
- is related to
-
SPARK-14493 "CREATE TEMPORARY TABLE ... USING ... AS SELECT ..." should always be used with a user defined path
- Resolved
- links to