[SPARK-44355] Move commands to CTEDef code path and deprecate CTE inline path

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Assignee: Wenchen Fan
• Reporter: Max Gekk
• Created: 10/Jul/23 17:11
• Updated: 25/Jul/23 13:54
• Resolved: 25/Jul/23 13:53

Description

Right now our CTE resolution code path is diverged: query with commands go into CTE inline code path where non-command queries go into CTEDef code path (see https://github.com/apache/spark/blob/42719d9425b9a24ef016b5c2874e522b960cf114/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CTESubstitution.scala#L50 ).

For longer term we should migrate command queries go through CTEDef as well and deprecate the CTE inline path.

Issue Links

• links to: [Github] Pull Request #42036 (cloud-fan)

Sub-Tasks

• Move INSERT INTO to CTEDef code path: Open