Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-14013 Support Flink Python User-Defined Stateless Function for Table
  3. FLINK-14014

Introduce PythonScalarFunctionRunner to handle the communication with Python worker for Python ScalarFunction execution

    XMLWordPrintableJSON

    Details

      Description

      PythonScalarFunctionRunner is responsible for Python ScalarFunction execution and it only handles the Python ScalarFunction execution and nothing else. So its logic should be very simple, forwarding an input element to Python worker and fetching the execution results back:

      1. Internally, it uses Apache Beam’s portability for Python UDF execution and this is transparent for the caller of PythonScalarFunctionRunner
      2. By default, each runner will startup a separate Python worker
      3. The Python worker can run in a docker, a separate process or even an non-managed external service.
      4. It has the ability to execute multiple Python ScalarFunctions
      5. It also supports chained Python ScalarFunctions

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                dian.fu Dian Fu
                Reporter:
                dian.fu Dian Fu
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m