[SPARK-42035] Add a config flag to force exit on JDK major version mismatch

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 3.4.0
• Component/s: Spark Core
• Target Version/s: 4.0.0
• Assignee: Holden Karau
• Reporter: Holden Karau
• Created: 12/Jan/23 18:00
• Updated: 09/Aug/23 19:16

Description

JRE version mismatches can cause errors which are difficult to debug (potentially correctness with serialization issues). We should add a flag for platform which wish to "fail fast" and exit on major version mismatch.

 

I think this could be a good thing to have default on in Spark 4.

 

Generally I expect to see more folks upgrading JRE & JDKs in the coming few years.

Issue Links

• is related to: SPARK-42036 Kryo ClassCastException getting task result when JDK versions mismatch