[SPARK-29909] Drop Python 2 and Python 3.4 and 3.5.

• Type: Umbrella
• Status: Resolved
• Priority: Major
• Resolution: Done
• Affects Version/s: 3.1.0
• Component/s: PySpark
• Labels: release-notes
• Target Version/s: 3.1.0
• Assignee: Hyukjin Kwon
• Reporter: Hyukjin Kwon
• Created: 15/Nov/19 04:47
• Updated: 12/Dec/22 17:51
• Resolved: 19/Jul/20 02:03

Description

We deprecated PySpark at SPARK-27884. We should drop at Spark 3.1.

Issue Links

• relates to: SPARK-27884 Deprecate Python 2 and Python 3 prior to 3.6 support in Spark 3.0

Sub-Tasks

• Update remaining python scripts in repo to python3 shebang: Resolved
• Remove python2 test execution in Jenkins environment: Resolved
• Drop Python 2, 3.4 and 3.5 in codes and documentation: Resolved
• Remove heapq3 port from Python 3: Resolved