[FLINK-15366] Improve FlinkCalcMergeRule to merge calc nodes better

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.12.0
• Component/s: Table SQL / Planner
• Labels: pull-request-available
• Assignee: Leonard Xu
• Reporter: Leonard Xu
• Created: 24/Dec/19 03:00
• Updated: 04/Aug/20 07:15
• Resolved: 28/Jul/20 02:45

Description

FlinkCalcMergeRule should merge calc nodes when outer calc do not contains inner calc's fields, the following logical plan can not merge as expected currently:

FlinkLogicalJoin(condition=[AND(=($0, $4), >($1, 1))], joinType=[left])
      :- FlinkLogicalCalc(select=[id, len, content], where=[>(id, 1)])
      :  +- FlinkLogicalTableSourceScan(table=[[default_catalog, default_database, T]], fields=[id, len, content])
      +- FlinkLogicalSnapshot(period=[$cor0.proctime])
         +- FlinkLogicalCalc(select=[age, id, name], where=[AND(>(age, 20), =(name, _UTF-16LE'Fabian':VARCHAR(2147483647) CHARACTER SET "UTF-16LE"))])
            +- FlinkLogicalCalc(select=[age, id, name, PROCTIME() AS proctime])
               +- FlinkLogicalTableSourceScan(table=[[default_catalog, default_database, userTable]], fields=[age, id, name])

The corresponding SQL to reproduce this issue:

 CREATE TABLE userTable (
       `id` BIGINT,
       `len` BIGINT,
       `content` STRING,
       `proctime` AS PROCTIME()
       ) WITH (
       'connector' = 'values',
       'data-id' = '$dataId',
       'bounded' = 'true'
 )

SELECT T.id, T.len, T.content, D.name FROM T JOIN userTable " +
  "for system_time as of T.proctime AS D ON T.id = D.id

 

Issue Links

• Is contained by: FLINK-16780 improve Flink lookup join
• links to: GitHub Pull Request #12873