[SPARK-38886] Remove outer join if aggregate functions are duplicate agnostic on streamed side

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: SQL
• Assignee: XiDuo You
• Reporter: XiDuo You
• Created: 13/Apr/22 07:05
• Updated: 18/Apr/22 15:23
• Resolved: 18/Apr/22 15:22

Description

If aggregate child is outer join, and the aggregate references are all coming from the streamed side and the aggregate functions are all duplicate agnostic, we can remve the outer join.

For example:

SELECT t1.c1, min(t1.c2) FROM t1 LEFT JOIN t2 ON t1.c1 = t2.c1 GROUP BY t1.c1
==>
SELECT t1.c1, min(t1.c2) FROM t1 GROUP BY t1.c1

Issue Links

• links to: [Github] Pull Request #36177 (ulysses-you)
• links to: [Github] Pull Request #36177 (ulysses-you)