[SPARK-46625] IDENTIFIER clause does not work with CTE reference

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.4
• Fix Version/s: 4.0.0
• Component/s: Spark Core
• Labels: pull-request-available
• Assignee: Nebojsa Savic
• Reporter: Serge Rielau
• Shepherd: Wenchen Fan
• Created: 08/Jan/24 17:51
• Updated: 09/Jul/24 14:31
• Resolved: 09/Jul/24 14:31

Description

IDENTIFIER clause does not pick up CTE 

DECLARE agg = 'max';
DECLARE col = 'c1';
DECLARE tab = 'T';

WITH S(c1, c2) AS (VALUES(1, 2), (2, 3)),
      T(c1, c2) AS (VALUES ('a', 'b'), ('c', 'd'))
SELECT IDENTIFIER(agg)(IDENTIFIER(col)) FROM IDENTIFIER(tab);

[TABLE_OR_VIEW_NOT_FOUND] The table or view `T` cannot be found. Verify the spelling and correctness of the schema and catalog.

If you did not qualify the name with a schema, verify the current_schema() output, or qualify the name with the correct schema and catalog.

To tolerate the error on drop use DROP VIEW IF EXISTS or DROP TABLE IF EXISTS. SQLSTATE: 42P01; line 3 pos 45;

'Project [unresolvedalias(expressionwithunresolvedidentifier('agg, org.apache.spark.sql.catalyst.parser.AstBuilder$$Lambda$2785/0x0000009002071490@126688a7))]

+- 'UnresolvedRelation [T], [], false

Issue Links

• links to: GitHub Pull Request #47180