[SPARK-41290] Support GENERATED ALWAYS AS syntax in create/replace table to create a generated column

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Allison Portis
• Reporter: Allison Portis
• Created: 28/Nov/22 04:48
• Updated: 05/Jul/23 22:38
• Resolved: 27/Feb/23 08:42

Description

Support GENERATED ALWAYS AS syntax for defining generated columns in create table and replace table.

For example,

CREATE TABLE default.example (
    time TIMESTAMP,
    date DATE GENERATED ALWAYS AS (CAST(time AS DATE))
)

 

This syntax is SQL standard and will enable defining generated columns in Spark SQL for data sources that support it.

 

Issue Links

• is related to: SPARK-44313 Generated column expression validation fails if there is a char/varchar column anywhere in the schema
• links to: [Github] Pull Request #38823 (allisonport-db)
• links to: [Github] Pull Request #38823 (allisonport-db)