[SPARK-29584] NOT NULL is not supported in Spark

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 3.0.0
• Component/s: SQL
• Assignee: Unassigned
• Reporter: ABHISHEK KUMAR GUPTA
• Created: 24/Oct/19 04:34
• Updated: 29/Dec/19 09:13
• Resolved: 29/Dec/19 09:13

Description

Spark while creating table restricting column for NULL value is not supported.

As below

PostgreSQL: SUCCESS No Exception

CREATE TABLE Persons (ID int NOT NULL, LastName varchar(255) NOT NULL,FirstName varchar(255) NOT NULL, Age int);
insert into Persons values(1,'GUPTA','Abhi',NULL);
select * from persons;

 

Spark: Parse Exception

jdbc:hive2://10.18.19.208:23040/default> CREATE TABLE Persons (ID int NOT NULL, LastName varchar(255) NOT NULL,FirstName varchar(255) NOT NULL, Age int);
Error: org.apache.spark.sql.catalyst.parser.ParseException:
no viable alternative at input 'CREATE TABLE Persons (ID int NOT'(line 1, pos 29)

 Parse Exception

Issue Links

• is duplicated by: SPARK-19842 Informational Referential Integrity Constraints Support in Spark