[SPARK-37716] Allow LateralJoin node to host non-deterministic expressions when the outer query is a single row relation

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.1
• Fix Version/s: 3.3.0
• Component/s: SQL
• Assignee: Allison Wang
• Reporter: Allison Wang
• Created: 22/Dec/21 18:28
• Updated: 27/Dec/21 09:04
• Resolved: 27/Dec/21 09:04

Description

After https://issues.apache.org/jira/browse/SPARK-37199, Analyzer will block LateralJoin that has non-deterministic lateral subqueries. But when the outer query is a single row relation, this should be allowed

For example:
Query:

SELECT * FROM VALUES(0) t(x) JOIN LATERAL (SELECT rand(0) + x AS y); 

Result:

org.apache.spark.sql.AnalysisException: nondeterministic expressions are only allowed in
Project, Filter, Aggregate or Window

 

Issue Links

• links to: [Github] Pull Request #34987 (allisonwang-db)