[SPARK-48033] Support Generated Column expressions that are `RuntimeReplaceable`

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Richard Chen
• Reporter: Richard Chen
• Created: 29/Apr/24 00:45
• Updated: 30/Apr/24 01:07
• Resolved: 30/Apr/24 01:07

Description

Currently, default columns that have a default of a `RuntimeReplaceable` expression fails.

This is because the `AlterTableCommand` constant folds before replacing expressions with the actual implementation. For example:
```
sql(s"CREATE TABLE t(v VARIANT DEFAULT parse_json('1')) USING PARQUET")
sql("INSERT INTO t VALUES(DEFAULT)")
```

fails because `parse_json` is `RuntimeReplaceable` and is evaluated before the analyzer inserts the correct expression into the plan

This is especially important for Variant types because literal variants are difficult to create - `parse_json` will likely be used the majority of the time.

Issue Links

• links to: GitHub Pull Request #46269