[SPARK-49063] Fix Between with ScalarSubqueries

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: SQL
• Labels: pull-request-available
• Assignee: Mihailo Milosevic
• Reporter: Mihailo Milosevic
• Created: 30/Jul/24 21:18
• Updated: 05/Aug/24 08:15
• Resolved: 05/Aug/24 08:15

Description

https://issues.apache.org/jira/browse/SPARK-46366 introduced a regression where types of queries like:

SELECT *
FROM t1
WHERE (select max(t2.t2c)
       from t2 where t1.t1b = t2.t2b
      ) between 1 and 20; 

would fail in optimizer rule.

Issue Links

• links to: GitHub Pull Request #47581