[SPARK-31136] Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax

• Type: Sub-task
• Status: Closed
• Priority: Blocker
• Resolution: Won't Do
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Dongjoon Hyun
• Created: 12/Mar/20 16:56
• Updated: 13/May/20 20:04
• Resolved: 20/Mar/20 05:13

Description

We need to consider the behavior change of SPARK-30098 .
This is a placeholder to keep the discussion and the final decision.

`CREATE TABLE` syntax changes its behavior silently.

The following is one example of the breaking the existing user data pipelines.
Apache Spark 2.4.5

spark-sql> CREATE TABLE t(a STRING);

spark-sql> LOAD DATA INPATH '/usr/local/spark/README.md' INTO TABLE t;

spark-sql> SELECT * FROM t LIMIT 1;
# Apache Spark
Time taken: 2.05 seconds, Fetched 1 row(s)

spark-sql> CREATE TABLE t(a CHAR(3));

spark-sql> INSERT INTO TABLE t SELECT 'a ';

spark-sql> SELECT a, length(a) FROM t;
a  	3

Apache Spark 3.0.0-preview2

spark-sql> CREATE TABLE t(a STRING);

spark-sql> LOAD DATA INPATH '/usr/local/spark/README.md' INTO TABLE t;
Error in query: LOAD DATA is not supported for datasource tables: `default`.`t`;

spark-sql> CREATE TABLE t(a CHAR(3));

spark-sql> INSERT INTO TABLE t SELECT 'a ';

spark-sql> SELECT a, length(a) FROM t;
a 	2

Issue Links

• is caused by: SPARK-30098 Add a configuration to use default datasource as provider for CREATE TABLE command
• is cloned by: SPARK-31707 Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax
• relates to: SPARK-31147 forbid CHAR type in non-Hive-Serde tables
• relates to: SPARK-31181 Remove the default value assumption on CREATE TABLE test cases
• relates to: SPARK-31133 fix sql ref doc for DML
• links to: [Github] Pull Request #28517 (HeartSaVioR)
• links to: [Github] Pull Request #28517 (HeartSaVioR)
• links to: GitHub Pull Request #27894