[SPARK-28662] Create Hive Partitioned Table without specifying data type for partition columns will success unexpectedly

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: SQL
• Target Version/s: 3.0.0
• Assignee: Li Hao
• Reporter: Li Hao
• Created: 08/Aug/19 13:38
• Updated: 20/Aug/19 06:42
• Resolved: 20/Aug/19 06:38

Description

Case :
Create Hive Partitioned Table without specifying data type for partition column will success unexpectly.

// create a hive table partition by b, but the data type of b isn't specified.
CREATE TABLE tbl(a int) PARTITIONED BY (b) STORED AS parquet

 

Root Cause:

In https://issues.apache.org/jira/browse/SPARK-26435 ,  PARTITIONED BY clause  are extended to support Hive CTAS as following:

// Before
(PARTITIONED BY '(' partitionColumns=colTypeList ')’

//After
(PARTITIONED BY '(' partitionColumns=colTypeList ‘)’|
PARTITIONED BY partitionColumnNames=identifierList) |

Create Table Statement like above case will pass the syntax check,  and recognized as (PARTITIONED BY partitionColumnNames=identifierList) 。

We should check this case in visitCreateHiveTable and give a explicit error message to user

 

Issue Links

• links to: GitHub Pull Request #25390