[SPARK-48106] Use `Python 3.11` in `pyspark` tests of `build_and_test.yml`

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.0.0
• Fix Version/s: 4.0.0
• Component/s: Project Infra
• Labels: pull-request-available
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 03/May/24 03:23
• Updated: 03/May/24 06:35
• Resolved: 03/May/24 06:34

Description

• https://docs.python.org/3/whatsnew/3.11.html#summary-release-highlights

Python 3.11 is between 10-60% faster than Python 3.10. On average, we measured a 1.25x speedup on the standard benchmark suite.

Issue Links

• links to: GitHub Pull Request #46353