Details
-
Epic
-
Status: Reopened
-
Major
-
Resolution: Unresolved
-
4.0.0
-
None
-
Sql Scripting
Description
Design doc for this feature is in attachment.
High level example of Sql Script:
```
BEGIN
DECLARE c INT = 10;
WHILE c > 0 DO
INSERT INTO tscript VALUES (c);
SET c = c - 1;
END WHILE;
END
```
High level motivation behind this feature:
SQL Scripting gives customers the ability to develop complex ETL and analysis entirely in SQL. Until now, customers have had to write verbose SQL statements or combine SQL + Python to efficiently write business logic. Coming from another system, customers have to choose whether or not they want to migrate to pyspark. Some customers end up not using Spark because of this gap. SQL Scripting is a key milestone towards enabling SQL practitioners to write sophisticated queries, without the need to use pyspark. Further, SQL Scripting is a necessary step towards support for SQL Stored Procedures, and along with SQL Variables (released) and Temp Tables (in progress), will allow for more seamless data warehouse migrations.
Attachments
Attachments
Issue Links
- is related to
-
SPARK-44111 Prepare Apache Spark 4.0.0
-
- Open
-