[SPARK-40532] Python version for UDF should follow the servers version

• Type: Sub-task
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: Connect
• Assignee: Hyukjin Kwon
• Reporter: Martin Grund
• Created: 22/Sep/22 12:59
• Updated: 07/Feb/23 08:32
• Resolved: 07/Feb/23 08:32

Description

Currently, we artificially pin the Python version to 3.9 in the UDF generation code, but this should actually be the correct server vs client version.

 

In addition the version should be configured as part of the function definition proto message.

Issue Links

• links to: [Github] Pull Request #39914 (HyukjinKwon)