Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task SPARK-48566

SPARK-43797 [Bug] Partition indices are incorrect when UDTF analyze() uses both select and partitionColumns

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-48180

SPARK-43797 Analyzer bug with multiple ORDER BY items for input table argument

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-47976

SPARK-43797 Support running Python UDTF 'analyze' method from Spark executors

Unassigned Daniel Major Resolved Won't Do  
Sub-task SPARK-47214

SPARK-43797 Create API for 'analyze' method to differentiate constant NULL arguments and other types of arguments

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-47032

SPARK-43797 Create API for 'analyze' method to send input column(s) to output table unchanged

Unassigned Daniel Major Resolved Won't Fix  
Sub-task SPARK-47002

SPARK-43797 Enforce that 'AnalyzeResult' 'orderBy' field is a list of pyspark.sql.functions.OrderingColumn

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-46966

SPARK-43797 Create API for 'analyze' method to indicate subset of input table columns to select

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-46638

SPARK-43797 Create API to acquire execution memory for 'eval' and 'terminate' methods

Unassigned Daniel Major Closed Won't Fix  
Sub-task SPARK-46040

SPARK-43797 Update API for 'analyze' partitioning/ordering columns to support general expressions

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-45810

SPARK-43797 Create API to stop consuming rows from the input table

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-45746

SPARK-43797 Return specific error messages if UDTF 'analyze' method accepts or returns wrong values

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-45523

SPARK-43797 Return useful error message if UDTF returns None for non-nullable column

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-45505

SPARK-43797 Refactor analyzeInPython function to make it reusable

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-45402

SPARK-43797 Add API for 'analyze' method to return a buffer to be consumed on each class creation

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-45401

SPARK-43797 Add a new method `cleanup` in the UDTF interface

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-45362

SPARK-43797 Project out PARTITION BY expressions before 'eval' method consumes input rows

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-44901

SPARK-43797 Add API in 'analyze' method to return partitioning/ordering expressions

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-44856

SPARK-43797 Improve Python UDTF arrow serializer performance

Michael Zhang Allison Wang Major Open Unresolved  
Sub-task SPARK-44836

SPARK-43797 Refactor Arrow Python UDTF

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44834

SPARK-43797 Add SQL query test suites for Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44822

SPARK-43797 Make Python UDTFs by default non-deterministic

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44766

SPARK-43797 Cache the pandas converter for Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44749

SPARK-43797 Support named arguments in Python UDTF

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44748

SPARK-43797 Query execution to support PARTITION BY and ORDER BY clause for table arguments

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-44746

SPARK-43797 Improve the documentation for TABLE input arguments for UDTFs

Daniel Allison Wang Major Resolved Fixed  
Sub-task SPARK-44663

SPARK-43797 Disable arrow optimization by default for Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44648

SPARK-43797 Set up memory limits for analyze in Python.

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44644

SPARK-43797 Improve error messages for creating Python UDTFs with pickling errors

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44640

SPARK-43797 Improve error messages for Python UDTF returning non iterable

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44561

SPARK-43797 Fix AssertionError when converting UDTF output to a complex type

Takuya Ueshin Allison Wang Major Resolved Fixed  
Sub-task SPARK-44559

SPARK-43797 Improve error messages for Python UDTF arrow type casts

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44533

SPARK-43797 Add support for accumulator, broadcast, and Spark files in Python UDTF's analyze.

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44508

SPARK-43797 Add user guide for Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-44503

SPARK-43797 Query planning to support PARTITION BY and ORDER BY clause for table arguments

Daniel Daniel Major Resolved Fixed  
Sub-task SPARK-44479

SPARK-43797 Support Python UDTFs with empty schema

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44380

SPARK-43797 Support for UDTF to analyze in Python

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44249

SPARK-43797 Refactor PythonUDTFRunner to send its return type separately

Takuya Ueshin Takuya Ueshin Major Resolved Fixed  
Sub-task SPARK-44009

SPARK-43797 Support profiler for Python UDTFs

Unassigned Allison Wang Major Open Unresolved  
Sub-task SPARK-44008

SPARK-43797 Include the name of the UDTF in the error messages generated during the function execution

Unassigned Allison Wang Major Open Unresolved  
Sub-task SPARK-44005

SPARK-43797 Improve error messages for regular Python UDTFs that return non-tuple values

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-43968

SPARK-43797 Improve error messages for Python UDTFs with wrong number of outputs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-43967

SPARK-43797 Support Python UDTFs with empty return values

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-43966

SPARK-43797 Support non-deterministic Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-43965

SPARK-43797 Support Python UDTFs in Spark Connect

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-43964

SPARK-43797 Support arrow-optimized Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  
Sub-task SPARK-43798

SPARK-43797 Initial support for Python UDTFs

Allison Wang Allison Wang Major Resolved Fixed  

Cancel