[FLINK-24048] Move changeLog inference out of optimizing phase

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.14.0
• Fix Version/s: 2.0.0
• Component/s: Table SQL / Planner
• Assignee: Unassigned
• Reporter: Shuo Cheng
• Created: 30/Aug/21 06:16
• Updated: 10/Jul/24 06:57

Description

Currently, when there are multiple sinks in a sql job, the DAG is split into multiple relNode blocks; as changeLog inference is in optimizing phase, we need to propagate the changeLog mode among blocks to ensure each block can generate an accurate physical plan.

In current solution, the DAG is optimized 3 times in order to propagate changeLog mode, which is inefficient. Actually, we can just optimize the DAG, expanding the DAG to a physical node tree, and then infer changeLog mode. In this way, the dag is only optimized 1 time.

(Similarly, minibatch interval can also be inferred in same way)

Issue Links

• mentioned in: Page Loading...