[SPARK-43831] Build and Run Spark on Java 21

• Type: New Feature
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.5.0
• Fix Version/s: 4.0.0
• Component/s: Build
• Labels: releasenotes
• Target Version/s: 4.0.0
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 28/May/23 00:22
• Updated: 20/Sep/23 19:47
• Resolved: 20/Sep/23 19:46

Description

• https://docs.scala-lang.org/overviews/jdk-compatibility/overview.html

  JDK version	Minimum Scala versions
  21 (ea)	3.3.1 (soon), 2.13.11, 2.12.18
  20	3.3.0, 2.13.11, 2.12.18
  19	3.2.0, 2.13.9, 2.12.16
  18	3.1.3, 2.13.7, 2.12.15
  17 (LTS)	3.0.0, 2.13.6, 2.12.15
  11 (LTS)	3.0.0, 2.13.0, 2.12.4, 2.11.12
  8 (LTS)	3.0.0, 2.13.0, 2.12.0, 2.11.0

Issue Links

• is related to: SPARK-40732 Make (SQL|ThriftServer)QueryTestSuite pass except Pandas UDF tests in Java 21
• is related to: SPARK-44111 Prepare Apache Spark 4.0.0
• relates to: ORC-1426 Use Java 21-ea instead of 20 in GitHub Action
• links to: Apache Arrow Java 21 Fix
• links to: Scala-Java Compatibility