[FLINK-14013] Support Flink Python User-Defined Stateless Function for Table

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.10.0
• Fix Version/s: 1.10.0
• Component/s: API / Python, Table SQL / API
• Assignee: Unassigned
• Reporter: Dian Fu
• Created: 09/Sep/19 13:04
• Updated: 16/Jan/20 09:20
• Resolved: 16/Jan/20 09:20

Description

The Python Table API has been supported in release 1.9.0. See the FLIP-38 and FLINK-12308 for details. However, currently Python user-defined functions are still not supported. In this FLIP, we want to support stateless Python user-defined functions in Python Table API.

More detailed description could be found in FLIP-58.

The discussion could be found in mailing thread.

Issue Links

• is related to: FLINK-14500 Support Flink Python User-Defined Stateless Function for Table - Phase 2

Sub-Tasks

• Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution: Closed
• Introduce PythonScalarFunctionOperator as a standalone StreamOperator for Python ScalarFunction execution: Closed
• Introduce RelNodes FlinkLogicalPythonScalarFunctionExec and DataStreamPythonScalarFunctionExec which are containers for Python PythonScalarFunctions: Closed
• Support to start up Python worker in process mode: Closed
• Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work: Closed
• Python environment and dependency management: Closed
• Add rules to push down the Python ScalarFunctions contained in the join condition of Correlate node: Closed
• Add validation check for places where Python ScalarFunction cannot be used: Closed
• Manage the resource of Python worker properly: Closed
• Add documentation for Python user-defined functions: Closed
• Optimize the execution plan for Python Calc when there is a condition: Closed
• Optimize Python UDFs with parameters of constant values: Closed
• Support Python UDFs without arguments: Closed
• Support Blink planner for Python UDF: Closed
• Remove method FunctionDefinition#getLanguage: Closed
• Support all the data types in Python user-defined functions: Closed
• Improve the release script for Python API release package: Closed
• Improve the README.md in pyflink to prepare for PyPI release: Closed
• Support to run Python UDF jobs in a YARN cluster: Closed