[SPARK-24602] In Spark SQL, ALTER TABLE--CHANGE column1 column2 datatype is not supported in 2.3.1

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Incomplete
• Affects Version/s: 2.3.0
• Component/s: SQL
• Labels: bulk-closed

Environment

OS: SUSE11

Spark Version: 2.3

• Assignee: Unassigned
• Reporter: Sushanta Sen
• Created: 20/Jun/18 08:20
• Updated: 25/May/21 01:54
• Resolved: 25/May/21 01:39

Description

Precondition:

Spark cluster 2.3 is up and running

Test Steps:

1. Launch Spark-sql
2. spark-sql> CREATE TABLE t1(a int,string)                                                                 0: jdbc:hive2://ha-cluster/default> alter table t1 change a a1 int;
   Error: org.apache.spark.sql.AnalysisException: ALTER TABLE CHANGE COLUMN is not supported for changing column 'a' with type 'IntegerType' to 'b' with type 'IntegerType'; (state=,code=0)
3. Launch hive beeliine
4. repeat step1 & 2
5. 0: jdbc:hive2://10.18.108.126:10000/> desc del1;
   -----------------------------+

   col_name

   data_type

   comment

   -----------------------------+

   a1	int
   dob	int

   -----------------------------+
   2 rows selected (1.572 seconds)
   0: jdbc:hive2://10.18.108.126:10000/> alter table del1 change a1 a bigint;
   No rows affected (0.425 seconds)
   0: jdbc:hive2://10.18.108.126:10000/> desc del1;
   -----------------------------+

   col_name

   data_type

   comment

   -----------------------------+

   a	bigint
   dob	int

   -----------------------------+
   2 rows selected (0.364 seconds)

 

Actual Result: In spark sql, alter table change is not supported, whereas in hive beeline it is working fine.

Expected Result: ALTER Table CHANGE should be supported in Spark-SQL as well

Issue Links

• is duplicated by: SPARK-26521 Sparksql cannot modify the field name of a table
• is related to: SPARK-26519 spark sql CHANGE COLUMN not working