[IGNITE-12159] Ignite spark doesn't support Alter Column syntax

• Type: Task
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 2.8
• Fix Version/s: 2.8
• Component/s: spark
• Assignee: Alexey Zinoviev
• Reporter: Andrei Aleksandrov
• Created: 10/Sep/19 14:46
• Updated: 14/Jan/20 09:09
• Resolved: 10/Oct/19 14:42

Description

Steps:

1)Start the server
2)Run next SQL commands

CREATE TABLE person (id LONG, name VARCHAR(64), age LONG, city_id DOUBLE, zip_code LONG, PRIMARY KEY (name)) WITH "backups=1"
ALTER TABLE person ADD COLUMN (first_name VARCHAR(64), last_name VARCHAR(64))

3)After that run next spark code:

       String configPath = "client.xml";
      
       SparkConf sparkConf = new SparkConf()
               .setMaster("local")
               .setAppName("Example"); 

      IgniteSparkSession.builder()
               .appName("Spark Ignite catalog example")
               .master("local")
               .config("ignite.disableSparkSQLOptimization", true)
               .igniteConfig(configPath)
               .getOrCreate();
 
      Dataset<Row> df2 = igniteSession.sql("select * from person");       
      df2.show();

The result will contain only 5 columns from CREATE TABLE call.

http://apache-ignite-users.70518.x6.nabble.com/Altered-sql-table-adding-new-columns-does-not-reflect-in-Spark-shell-td29265.html

Issue Links

• duplicates: IGNITE-10314 Spark dataframe will get wrong schema if user executes add/drop column DDL