[FLINK-31273] Left join with IS_NULL filter be wrongly pushed down and get wrong join results

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.17.0, 1.16.1
• Fix Version/s: 1.16.2, 1.18.0, 1.17.1
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Yunhong Zheng
• Reporter: Yunhong Zheng
• Created: 01/Mar/23 07:27
• Updated: 28/Jun/23 14:35
• Resolved: 15/Mar/23 10:14

Description

Left join with IS_NULL filter be wrongly pushed down and get wrong join results. The sql is:

SELECT * FROM MyTable1 LEFT JOIN MyTable2 ON a1 = a2 WHERE a2 IS NULL AND a1 < 10

The wrongly plan is:

LogicalProject(a1=[$0], b1=[$1], c1=[$2], b2=[$3], c2=[$4], a2=[$5])
+- LogicalFilter(condition=[IS NULL($5)])
   +- LogicalJoin(condition=[=($0, $5)], joinType=[left])
      :- LogicalValues(tuples=[[]])
      +- LogicalTableScan(table=[[default_catalog, default_database, MyTable2]]) 

Issue Links

• is related to: FLINK-32471 IS_NOT_NULL can add to SUITABLE_FILTER_TO_PUSH
• links to: GitHub Pull Request #22049