[SPARK-29591] Support data insertion in a different order if you wish or even omit some columns in spark sql also like postgresql

• Type: Sub-task
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.4.4
• Component/s: SQL
• Assignee: Unassigned
• Reporter: jobit mathew
• Created: 24/Oct/19 13:57
• Updated: 22/Dec/20 10:21

Description

Support data insertion in a different order if you wish or even omit some columns in spark sql also like postgre sql.

In postgre sql

CREATE TABLE weather (
 city varchar(80),
 temp_lo int, – low temperature
 temp_hi int, – high temperature
 prcp real, – precipitation
 date date
 );

You can list the columns in a different order if you wish or even omit some columns,

INSERT INTO weather (date, city, temp_hi, temp_lo)
 VALUES ('1994-11-29', 'Hayward', 54, 37);

 

Spark SQL

But in spark sql is not allowing to insert data in different order or omit any column.Better to support this as it can save time if we can not predict any specific column value or if some value is fixed always.

create table jobit(id int,name string);

> insert into jobit values(1,"Ankit");
 Time taken: 0.548 seconds
 spark-sql> *insert into jobit (id) values(1);*
 *Error in query:*
 mismatched input 'id' expecting \{'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP', 'REDUCE'}(line 1, pos 19)

== SQL ==
 insert into jobit (id) values(1)
 -------------------^^^

spark-sql> *insert into jobit (name,id) values("Ankit",1);*
 *Error in query:*
 mismatched input 'name' expecting \{'(', 'SELECT', 'FROM', 'VALUES', 'TABLE', 'INSERT', 'MAP', 'REDUCE'}(line 1, pos 19)

== SQL ==
 insert into jobit (name,id) values("Ankit",1)
 -------------------^^^

spark-sql>