[SPARK-48388] [M0] Fix SET behavior for scripts

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: David Milicevic
• Reporter: David Milicevic
• Created: 22/May/24 10:11
• Updated: 19/Jul/24 02:25
• Resolved: 19/Jul/24 02:25

Description

By standard, SET is used to set variable value in SQL scripts.

On our end, SET is configured to work with some Hive configs, so the grammar is a bit messed up and for that reason it was decided to use SET VAR instead of SET to work with SQL variables.

This is not by standard and we should figure out the way to be able to use SET for SQL variables and forbid setting of Hive configs from SQL scripts.

 

For more details, design doc can be found in parent Jira item.

Issue Links

• links to: GitHub Pull Request #47272