[SPARK-36114] Support subqueries with correlated non-equality predicates

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.2.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: Allison Wang
• Reporter: Allison Wang
• Created: 13/Jul/21 04:34
• Updated: 24/Oct/22 03:11
• Resolved: 24/Oct/22 03:11

Description

The new decorrelation framework is able to support subqueries with non-equality predicates. For example:

SELECT * FROM t1 WHERE c1 = (SELECT SUM(c1) FROM t2 WHERE t1.c2 > t2.c2)

The restrictions in CheckAnlysis can be removed.

 

Issue Links

• is related to: SPARK-35080 Correlated subqueries with equality predicates can return wrong results
• links to: [Github] Pull Request #38135 (allisonwang-db)
• links to: [Github] Pull Request #38135 (allisonwang-db)