Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
0.8.0
-
None
Description
Hi,
While running first spark program in zeppelin, it is failing to create spark interpreter process. Below are the error details. please let me know how to fix this issue.
INFO [2018-07-29 00:06:05,323] ({pool-2-thread-2} Paragraph.java[jobRun]:380) - Run paragraph [paragraph_id: 20180728-222626_1797203208, interpreter: spark, note_id: 2DN1RQXVR, user: anonymous]
INFO [2018-07-29 00:06:05,323] ({pool-2-thread-2} ManagedInterpreterGroup.java[getOrCreateInterpreterProcess]:61) - Create InterpreterProcess for InterpreterGroup: spark:shared_process
INFO [2018-07-29 00:06:05,325] ({pool-2-thread-2} ShellScriptLauncher.java[launch]:48) - Launching Interpreter: spark
INFO [2018-07-29 00:06:05,351] ({pool-2-thread-2} SparkInterpreterLauncher.java[buildEnvFromProperties]:103) - Run Spark under non-secure mode as no keytab and principal is specified
DEBUG [2018-07-29 00:06:05,352] ({pool-2-thread-2} SparkInterpreterLauncher.java[buildEnvFromProperties]:105) - buildEnvFromProperties: {ZEPPELIN_SPARK_CONF= --master local[*] --conf spark.yarn.dist.archives=C:\spark\R\lib\sparkr.zip#sparkr --conf spark.app.name='Zeppelin', SPARK_HOME=C:\spark}
INFO [2018-07-29 00:06:05,361] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:115) - Thrift server for callback will start. Port: 57136
DEBUG [2018-07-29 00:06:05,371] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:153) - callbackServer is serving now
INFO [2018-07-29 00:06:05,380] ({pool-2-thread-2} RemoteInterpreterManagedProcess.java[start]:190) - Run interpreter process [C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all\bin\interpreter.cmd, -d, C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all/interpreter/spark, -c, 10.120.44.23, -p, 57136, -r, :, -l, C:\zeppelin\zeppelin-0.8.0-bin-all\zeppelin-0.8.0-bin-all/local-repo/spark, -g, spark]
DEBUG [2018-07-29 00:06:09,625] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - Warning: Local jar C:\Users\vv\57136 does not exist, skipping.
DEBUG [2018-07-29 00:06:09,626] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - Warning: Local jar C:\Users\vv\10.120.44.23 does not exist, skipping.
DEBUG [2018-07-29 00:06:09,627] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - java.lang.ClassNotFoundException: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
DEBUG [2018-07-29 00:06:09,628] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
DEBUG [2018-07-29 00:06:09,630] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
DEBUG [2018-07-29 00:06:09,632] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
DEBUG [2018-07-29 00:06:09,633] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.Class.forName0(Native Method)
DEBUG [2018-07-29 00:06:09,634] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at java.lang.Class.forName(Class.java:348)
DEBUG [2018-07-29 00:06:09,635] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
DEBUG [2018-07-29 00:06:09,641] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
DEBUG [2018-07-29 00:06:09,642] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
DEBUG [2018-07-29 00:06:09,644] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
DEBUG [2018-07-29 00:06:09,645] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
DEBUG [2018-07-29 00:06:09,647] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
DEBUG [2018-07-29 00:06:09,656] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - 2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Shutdown hook called
DEBUG [2018-07-29 00:06:09,659] ({Exec Stream Pumper} RemoteInterpreterManagedProcess.java[processLine]:298) - 2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Deleting directory C:\Users\vv\AppData\Local\Temp\spark-427c3202-c243-4761-86ce-ea51a27a881c
INFO [2018-07-29 00:06:09,747] ({Exec Default Executor} RemoteInterpreterManagedProcess.java[onProcessFailed]:250) - Interpreter process failed {}
org.apache.commons.exec.ExecuteException: Process exited with an error: 101 (Exit value: 101)
at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:404)
at org.apache.commons.exec.DefaultExecutor.access$200(DefaultExecutor.java:48)
at org.apache.commons.exec.DefaultExecutor$1.run(DefaultExecutor.java:200)
at java.lang.Thread.run(Thread.java:748)
ERROR [2018-07-29 00:07:05,382] ({pool-2-thread-2} Job.java[run]:190) - Job failed
java.lang.RuntimeException: Warning: Local jar C:\Users\vv\57136 does not exist, skipping.
Warning: Local jar C:\Users\vv\10.120.44.23 does not exist, skipping.
java.lang.ClassNotFoundException: org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:348)
at org.apache.spark.util.Utils$.classForName(Utils.scala:238)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:851)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:198)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:228)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:137)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
2018-07-29 00:06:09 INFO ShutdownHookManager:54 - Shutdown hook called
Thanks
VV