[SPARK-39791] In Spark 3.0 standalone cluster mode, unable to customize driver JVM path

• Type: Question
• Status: In Progress
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 3.0.0
• Component/s: Spark Submit
• Labels: spark-submit standalone
• Assignee: Unassigned
• Reporter: Obobj
• Created: 15/Jul/22 06:09
• Updated: 12/Dec/22 18:10

Description

In Spark 3.0 standalone mode, unable to customize driver JVM path, instead the JAVA_HOME of the spark-submit submission machine is used, but the JVM paths of my submission machine and the cluster machine are different

launcher/src/main/java/org/apache/spark/launcher/AbstractCommandBuilder.java

List<String> buildJavaCommand(String extraClassPath) throws IOException {
  List<String> cmd = new ArrayList<>();

  String firstJavaHome = firstNonEmpty(javaHome,
    childEnv.get("JAVA_HOME"),
    System.getenv("JAVA_HOME"),
    System.getProperty("java.home")); 

Issue Links

• links to: [Github] Pull Request #37578 (obobj)