[SPARK-43797] Python User-defined Table Functions

• Type: Umbrella
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.5.0, 4.0.0
• Component/s: PySpark
• Assignee: Unassigned
• Reporter: Allison Wang
• Created: 25/May/23 17:40
• Updated: 08/Jun/24 17:54

Description

This is an umbrella ticket to support Python user-defined table functions.

Issue Links

• relates to: SPARK-39247 Support returning a table or set of rows in CREATE FUNCTION

Sub-Tasks

• Include the name of the UDTF in the error messages generated during the function execution: Open
• Support profiler for Python UDTFs: Open
• Improve Python UDTF arrow serializer performance: Open