Details
-
New Feature
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
Description
As Table API will be the unified high level API for both batch and streaming, it is desired to have a table API based ML Pipeline definition.
This new table based ML Pipeline shall:
- support unified batch/stream ML/AI functionalities (train/inference).
- seamless integrated with flink Table based ecosystem.
- provide a base for further flink based AI platform/tooling support.
- provide a base for further flink ML SQL integration.
The initial design is here https://docs.google.com/document/d/1PLddLEMP_wn4xHwi6069f3vZL7LzkaP0MN9nAB63X90/edit?usp=sharing.
And based on this design, I made some initial implementation/prototyping. I will share the code later.
This is the umbrella JIRA. I will create corresponding sub-jira for each sub-task.
Attachments
Issue Links
- is duplicated by
-
FLINK-12470 FLIP39: Flink ML pipeline and ML libs
- Closed
1.
|
Create a new table based flink ML package | Closed | Unassigned | |
2.
|
Create a new set of table based ML Pipeline classes | Closed | Unassigned | |
3.
|
Create Table based optimizers | Closed | Unassigned | |
4.
|
Support pipeline stage type inference | Closed | Unassigned | |
5.
|
Create a new set of parameters | Closed | Unassigned | |
6.
|
Support pipeline import/export | Closed | Unassigned | |
7.
|
Support periodically update models when inferencing | Closed | Unassigned | |
8.
|
Support wrapping inference pipeline as a UDF function in SQL | Closed | Unassigned | |
9.
|
Port some flink.ml algorithms to table based | Closed | Unassigned |