[SEDONA-668] Drop the support of Spark 3.0, 3.1, 3.2

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: Jia Yu
• Created: 24/Oct/24 04:29
• Updated: 24/Nov/24 07:59
• Resolved: 24/Nov/24 07:59

Description

Apache Spark has officially announced the EOL of Spark 3.0, 3.1, and 3.2 long time ago. It has been a huge overhead for us to support Sedona on all these versions. We will drop the support of Spark 3.0, 3.1, and 3.2

 

Here is a summary of current Spark versions on major Spark hosting services

 

	Spark 3.0	Spark 3.1	Spark 3.2	Spark 3.3
-------------------	---------------	-------------------	------------------	-------------------
Databricks	EOL	Dec 19, 2024	Mar 18, 2025	Mar 1, 2026
AWS Glue	EOL	Unknown, Glue 3.0	N/A	Unknown, Glue 4.0
AWS EMR	July 24, 2025	July 24, 2025	January 25, 2026	January 25, 2026
Microsoft Fabric	EOL	EOL	EOL	Unknown
Microsoft Synapse	EOL	EOL	EOL	3/31/2025
Google Dataproc	EOL	EOL	N/A	2024/12/31

Issue Links

• links to: GitHub Pull Request #1653