[SPARK-34191] udf type hint should allow dectorator with named returnType

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0, 3.1.1, 3.2.0
• Fix Version/s: 3.1.1
• Component/s: PySpark
• External issue URL: https://github.com/zero323/pyspark-stubs/pull/546
• Assignee: Piotr Grzegorski
• Reporter: Maciej Szymkiewicz
• Created: 21/Jan/21 14:08
• Updated: 12/Dec/22 18:10
• Resolved: 22/Jan/21 12:20

Description

At the moment annotations allow following decorated patterns:
 

@udf
def f(x): ...

@udf("string")  # Or DataType instance
def f(x): ...

@udf(f="string")  # Awkward but technically valid
def f(x): ...

We should also support

@udf(returnType="string")
def f(x): ...

Issue Links

• links to: [Github] Pull Request #31282 (pgrz)