[SPARK-27741] Transitivity on predicate pushdown

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.1.1
• Component/s: SQL
• Assignee: Unassigned
• Reporter: U Shaw
• Created: 16/May/19 07:40
• Updated: 12/Dec/22 18:11
• Resolved: 16/May/19 10:17

Description

When using inner join, where conditions can be passed to join on, and when using outer join, even if the conditions are the same, only the predicate is pushed down to left or right.
As follows:

select * from t1 left join t2 on t1.id=t2.id where t1.id=1
--> select * from t1 left join on t1.id=t2.id and t2.id=1 where t1.id=1

Is Catalyst can support transitivity on predicate pushdown ?

Issue Links

• duplicates: SPARK-21479 Outer join filter pushdown in null supplying table when condition is on one of the joined columns