[FLINK-19621] Introduce Multi-input operator in Blink planner

• Type: New Feature
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Component/s: Table SQL / Planner, Table SQL / Runtime
• Labels: auto-deprioritized-major
• Assignee: Unassigned
• Reporter: Caizhi Weng
• Created: 14/Oct/20 02:55
• Updated: 07/Sep/21 16:50
• Resolved: 16/Jul/21 07:57

Description

As runtime now supports multi-input tasks and source chaining, we're going to introduce multi-input operator in Blink planner to remove unnecessary shuffle, thus improve the performance for both batch and streaming.

Design doc: https://docs.google.com/document/d/1qKVohV12qn-bM51cBZ8Hcgp31ntwClxjoiNBUOqVHsI/edit

Issue Links

• mentioned in: Page Loading...

Sub-Tasks

• Introduce ExecEdge to describe information on input edges for ExecNode: Closed
• Update deadlock break-up algorithm to cover more cases: Closed
• Introduce multi-input exec node: Closed
• Introduce multi-input operator construction algorithm: Closed
• Introduce TableOperatorWrapperGenerator to translate transformation DAG in a multiple-input node to TableOperatorWrapper DAG: Closed
• Introduce multi-input operator for batch: Closed
• Use multi-input optimization by default: Closed