[FLINK-10972] Enhancements to Flink Table API

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.9.0
• Fix Version/s: 1.9.0
• Component/s: Table SQL / API
• Assignee: sunjincheng
• Reporter: sunjincheng
• Created: 22/Nov/18 01:27
• Updated: 29/Jul/19 12:28
• Resolved: 29/Jul/19 12:27

Description

link titleWith the continuous efforts from the community, the Flink system has been continuously improved, which has attracted more and more users. Flink SQL is a canonical, widely used relational query language. However, there are still some scenarios where Flink SQL failed to meet user needs in terms of functionality and ease of use, such as:

• In terms of functionality

Iteration, user-defined window, user-defined join, user-defined GroupReduce, etc. Users cannot express them with SQL;

• In terms of ease of use

• Map - e.g. “dataStream.map(mapFun)”. Although “table.select(udf1(), udf2(), udf3()....)” can be used to accomplish the same function., with a map() function returning 100 columns, one has to define or call 100 UDFs when using SQL, which is quite involved.

• FlatMap -  e.g. “dataStrem.flatmap(flatMapFun)”. Similarly, it can be implemented with “table.join(udtf).select()”. However, it is obvious that datastream is easier to use than SQL.

Due to the above two reasons, In this JIRAs group, we will enhance the TableAPI in stages.

-----------------------

The first stage we seek to support (will describe the details in the sub issue) :

• Table.map()
• Table.flatMap()
• GroupedTable.aggregate()
• GroupedTable.flatAggregate()

The FLIP can be find here: FLIP-29

 

The second part is about column operator/operations:

1)   Table(schema) operators

• Add columns
• Replace columns
• Drop columns
• Rename columns

2）Fine-grained column/row operations

• Column selection
• Row package and flatten

See google doc

 

 

Issue Links

• is related to: FLINK-13470 Enhancements to Flink Table API for blink planner
• mentioned in: Page Loading...

Sub-Tasks

• Add Map operator to Table API: Closed
• Add FlatMap to TableAPI: Closed
• Add support for TimeAttribute in Map/FlatMap operator: Closed
• Add Aggregate operator to Table API: Closed
• Add FlatAggregate operator to unbounded streaming Table API: Closed
• Add support for group keys in Unbounded Aggregate/FlatAggregate operator: Closed
• Add Bounded(Group Window) FlatAggregate operator to streaming Table API: Closed
• Add documentation for TableAggregate Function: Closed
• BatchTableEnvironment and StreamTableEnvironment should transparent to users: Closed
• Add Column Operators/Operations: Closed
• Add Column Operators(add/rename/drop): Closed
• Add Column selections: Closed
• Support incremental emit under AccRetract mode for non-window streaming FlatAggregate on Table API: Closed
• Add group window Aggregate operator to Table API: Closed