[ZEPPELIN-1211] (Windows) running interpreter on windows with SPARK_SUBMIT

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Component/s: Interpreters

Environment

Windows

• Assignee: Unassigned
• Reporter: Oleksiy Dyagilev
• Created: 19/Jul/16 16:03
• Updated: 21/Apr/17 09:12

Description

How to repro:

• on Windows
• create /conf/zeppelin-env.cmd
• define SPARK_HOME and SPARK_SUBMIT_OPTIONS with "--jars ..." there
• start Zeppelin and run some spark notes

Observed:

• Zeppelin fails with errors

The problem:
in interpeter.cmd
when SPARK_SUBMIT is defined it calls
"%SPARK_SUBMIT%" --class %ZEPPELIN_SERVER% --jars %CLASSPATH% --driver-java-options "Unable to render embedded object: File (JAVA_INTP_OPTS) not found." %SPARK_SUBMIT_OPTIONS% "%SPARK_APP_JAR%" %PORT%

I think it's wrong to use %CLASSPATH% for --jars.
--jars expects comma separated jars and CLASSPATH is colon separated. I fixed it replacing --jars with --driver-class-path (as it's in interepreter.sh)

Also, when SPARK_SUBMIT is not defined, it calls
"%ZEPPELIN_RUNNER%" Unable to render embedded object: File (JAVA_INTP_OPTS) not found. %ZEPPELIN_INTP_MEM% -cp %ZEPPELIN_CLASSPATH_OVERRIDES%;%CLASSPATH% %ZEPPELIN_SERVER% %PORT%

Note, that CLASSPATH will already include the content of ZEPPELIN_CLASSPATH_OVERRIDES. Find the line above
set CLASSPATH=%ZEPPELIN_CLASSPATH_OVERRIDES%;%ZEPPELIN_CLASSPATH%

Issue Links

• relates to: ZEPPELIN-2440 [Umbrella] Platform Compatibility Issues