[FLINK-3226] Translate optimized logical Table API plans into physical plans representing DataSet programs

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Fix Version/s: 1.1.0
• Component/s: Table SQL / API
• Assignee: Chengxiang Li
• Reporter: Fabian Hueske
• Created: 12/Jan/16 17:35
• Updated: 18/Mar/16 13:58
• Resolved: 18/Mar/16 13:58

Description

This issue is about translating an (optimized) logical Table API (see FLINK-3225) query plan into a physical plan. The physical plan is a 1-to-1 representation of the DataSet program that will be executed. This means:

• Each Flink RelNode refers to exactly one Flink DataSet or DataStream operator.
• All (join and grouping) keys of Flink operators are correctly specified.
• The expressions which are to be executed in user-code are identified.
• All fields are referenced with their physical execution-time index.
• Flink type information is available.
• Optional: Add physical execution hints for joins

The translation should be the final part of Calcite's optimization process.

For this task we need to:

• implement a set of Flink DataSet RelNodes. Each RelNode corresponds to one Flink DataSet operator (Map, Reduce, Join, ...). The RelNodes must hold all relevant operator information (keys, user-code expression, strategy hints, parallelism).
• implement rules to translate optimized Calcite RelNodes into Flink RelNodes. We start with a straight-forward mapping and later add rules that merge several relational operators into a single Flink operator, e.g., merge a join followed by a filter. Timo implemented some rules for the first SQL implementation which can be used as a starting point.
• Integrate the translation rules into the Calcite optimization process

Issue Links

• depends upon: FLINK-3282 Add FlinkRelNode interface.