Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14013

Support Flink Python User-Defined Stateless Function for Table

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.10.0
    • Fix Version/s: None
    • Labels:
      None

      Description

      The Python Table API has been supported in release 1.9.0. See the FLIP-38 and FLINK-12308 for details. However, currently Python user-defined functions are still not supported. In this FLIP, we want to support stateless Python user-defined functions in Python Table API.

      More detailed description could be found in FLIP-58.

      The discussion could be found in mailing thread.

        Attachments

          Issue Links

          1.
          Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution Sub-task Closed Dian Fu
          2.
          Introduce PythonScalarFunctionOperator as a standalone StreamOperator for Python ScalarFunction execution Sub-task Closed Dian Fu
          3.
          Introduce RelNodes FlinkLogicalPythonScalarFunctionExec and DataStreamPythonScalarFunctionExec which are containers for Python PythonScalarFunctions Sub-task Closed Dian Fu
          4.
          Support to start up Python worker in process mode Sub-task Closed Wei Zhong
          5.
          Add Python building blocks to make sure the basic functionality of Python ScalarFunction could work Sub-task Closed Dian Fu
          6.
          Python environment and dependency management Sub-task Open Wei Zhong
          7.
          Add rules to push down the Python ScalarFunctions contained in the join condition of Correlate node Sub-task Open Huang Xingbo
          8.
          Add validation check for places where Python ScalarFunction cannot be used Sub-task Closed Huang Xingbo
          9.
          Manage the resource of Python worker properly Sub-task Open Dian Fu  
          10.
          Add documentation for Python user-defined functions Sub-task Closed Wei Zhong
          11.
          Support to configure the log level of the Python user-defined functions Sub-task Open Unassigned  
          12.
          Optimize the execution plan for Python Calc when there is a condition Sub-task Closed Dian Fu
          13.
          Optimize Python UDFs with parameters of constant values Sub-task Closed Huang Xingbo
          14.
          Support Python UDFs without arguments Sub-task Closed Wei Zhong
          15.
          The close method does not work for Python UserDefinedFunction Sub-task Open Dian Fu  
          16.
          Support Blink planner for Python UDF Sub-task Closed Huang Xingbo
          17.
          Remove method FunctionDefinition#getLanguage Sub-task Closed Wei Zhong
          18.
          Support python UDFs with constant value of time interval types Sub-task Open Huang Xingbo  
          19.
          Support all the data types in Python user-defined functions Sub-task Open Huang Xingbo
          20.
          Improve the release script for Python API release package Sub-task Open Unassigned
          21.
          Improve the README.md in pyflink to prepare for PyPI release Sub-task Closed Wei Zhong
          22.
          Support to run Python UDF jobs in a YARN cluster Sub-task Open Unassigned

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dian.fu Dian Fu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h 10m
                  5h 10m