[SPARK-46683] Write a subquery generator that generates subqueries of different variations to increase testing coverage in this area

• Type: Test
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.1
• Fix Version/s: 4.0.0
• Component/s: Optimizer, SQL
• Labels: correctness pull-request-available testing
• Assignee: Andy Lam
• Reporter: Andy Lam
• Created: 11/Jan/24 22:02
• Updated: 02/Feb/24 03:23
• Resolved: 23/Jan/24 09:50

Description

There are a lot of subquery correctness issues, ranging from very old bugs to new ones that are being introduced due to work being done on subquery correlation optimization. This is especially in the areas of COUNT bugs and null behaviors.

To increase test coverage and robustness in this area, we want to write a subquery generator that writes variations of subqueries, producing SQL tests that also run against Postgres (from my work in SPARK-46179).

Issue Links

• links to: GitHub Pull Request #44599
• links to: GitHub Pull Request #44956