[FLINK-16187] Support Python ML Pipeline API

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Resolved
• Fix Version/s: 1.11.0
• Component/s: API / Python
• Assignee: Hequn Cheng
• Reporter: Hequn Cheng
• Created: 20/Feb/20 12:47
• Updated: 18/Mar/20 13:39
• Resolved: 18/Mar/20 13:39

Description

This is the umbrella Jira for FLIP96, which intends to support Python ML Pipeline API.

FLIP wiki page: https://cwiki.apache.org/confluence/display/FLINK/FLIP-96%3A+Support+Python+ML+Pipeline+API

FLIP-39 rebuilds the Flink ML pipeline on top of TableAPI and introduces a new set of Java APIs. As Python is widely used in ML areas, providing Python ML Pipeline APIs for Flink can not only make it easier to write ML jobs for Python users but also broaden the adoption of Flink ML. In this FLIP we propose to:

• Add Python pipeline API according to Java pipeline API(we will adapt the Python pipeline API if Java pipeline API changes).
• Support native Python Transformer/Estimator/Model, i.e., users can write not only Python Transformer/Estimator/Model wrappers for calling Java ones but also can write native Python Transformer/Estimator/Models.
• Ease of use. Support keyword arguments when defining parameters.

Issue Links

• mentioned in: Page Loading...

Sub-Tasks

• Add interfaces for MLEnvironment and MLEnvironmentFactory: Closed
• Add interfaces for Params, ParamInfo and WithParams: Closed
• Add interfaces for PipelineStage and Pipeline: Closed
• Make Pipeline Json compitable between Java and Python if all Pipelinestage are Java ones: Closed