[SPARK-48180] Analyzer bug with multiple ORDER BY items for input table argument

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0, 4.0.0, 3.5.1
• Fix Version/s: 4.0.0
• Component/s: PySpark
• Labels: pull-request-available
• Assignee: Daniel
• Reporter: Daniel
• Created: 07/May/24 21:54
• Updated: 09/May/24 23:44
• Resolved: 09/May/24 23:44

Description

Steps to reproduce:

 

from pyspark.sql.functions import udtf

@udtf(returnType="a: int, b: int")
class tvf:
  def eval(self, *args):
    yield 1, 2

 

SELECT * FROM tvf(
  TABLE(
    SELECT 1 AS device_id, 2 AS data_ds
    )
    WITH SINGLE PARTITION
    ORDER BY device_id, data_ds
 )

[UNSUPPORTED_SUBQUERY_EXPRESSION_CATEGORY.UNSUPPORTED_TABLE_ARGUMENT] Unsupported subquery expression: Table arguments are used in a function where they are not supported:
'UnresolvedTableValuedFunction [tvf], table-argument#338 [], 'data_ds, false
   +- Project 1 AS device_id#336, 2 AS data_ds#337
      +- OneRowRelation

Issue Links

• links to: GitHub Pull Request #46451