[SPARK-32316] Test PySpark with Python 3.8 in Github Actions as well

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.1.0
• Fix Version/s: 3.1.0
• Component/s: Project Infra, Tests
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 15/Jul/20 01:31
• Updated: 12/Dec/22 17:51
• Resolved: 15/Jul/20 03:44

Description

Currently, only PyPy3 and Python 3.6 are being tested with PySpark. We should test the latest version of Python as well because some optimizations can be only enabled with Python 3.8+. See also https://github.com/apache/spark/pull/29114

Issue Links

• is related to: SPARK-32363 Flaky pip installation test in Jenkins
• links to: [Github] Pull Request #29116 (HyukjinKwon)
• links to: [Github] Pull Request #29116 (HyukjinKwon)