Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-12159

Ignite spark doesn't support Alter Column syntax

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.8
    • Fix Version/s: 2.8
    • Component/s: spark
    • Labels:
      None

      Description

      Steps:

      1)Start the server
      2)Run next SQL commands

      CREATE TABLE person (id LONG, name VARCHAR(64), age LONG, city_id DOUBLE, zip_code LONG, PRIMARY KEY (name)) WITH "backups=1"
      ALTER TABLE person ADD COLUMN (first_name VARCHAR(64), last_name VARCHAR(64))

      3)After that run next spark code:

             String configPath = "client.xml";
            
             SparkConf sparkConf = new SparkConf()
                     .setMaster("local")
                     .setAppName("Example"); 

            IgniteSparkSession.builder()
                     .appName("Spark Ignite catalog example")
                     .master("local")
                     .config("ignite.disableSparkSQLOptimization", true)
                     .igniteConfig(configPath)
                     .getOrCreate();
       
            Dataset<Row> df2 = igniteSession.sql("select * from person");       
            df2.show();

      The result will contain only 5 columns from CREATE TABLE call.

      http://apache-ignite-users.70518.x6.nabble.com/Altered-sql-table-adding-new-columns-does-not-reflect-in-Spark-shell-td29265.html

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                zaleslaw Alexey Zinoviev
                Reporter:
                aealeksandrov Andrey Aleksandrov
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: