[TOREE-500] Failed to start Toree on Spark 2.4.2

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.3.0
• Fix Version/s: 0.5.0
• Component/s: Kernel
• Environment: * Docker Container(openjdk8) ; * Python 3.5
• Assignee: Unassigned
• Reporter: Masayuki Takahashi
• Created: 05/May/19 10:00
• Updated: 02/Aug/20 05:13
• Resolved: 02/Aug/20 05:13

Description

[I 04:07:48.726 NotebookApp] Kernel started: 1d46275c-71d6-4062-bc22-a98593d6c374
Starting Spark Kernel with SPARK_HOME=/spark
19/05/05 04:07:53 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
Exception in thread "main" java.lang.NoClassDefFoundError: scala/App$class
        at org.apache.toree.Main$.<init>(Main.scala:24)
        at org.apache.toree.Main$.<clinit>(Main.scala)
        at org.apache.toree.Main.main(Main.scala)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.spark.deploy.JavaMainApplication.start(SparkApplication.scala:52)
        at org.apache.spark.deploy.SparkSubmit.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:849)
        at org.apache.spark.deploy.SparkSubmit.doRunMain$1(SparkSubmit.scala:167)
        at org.apache.spark.deploy.SparkSubmit.submit(SparkSubmit.scala:195)
        at org.apache.spark.deploy.SparkSubmit.doSubmit(SparkSubmit.scala:86)
        at org.apache.spark.deploy.SparkSubmit$$anon$2.doSubmit(SparkSubmit.scala:924)
        at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:933)
        at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.lang.ClassNotFoundException: scala.App$class
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357)

Toree on Spark 2.4.1 works for me.

See: https://github.com/jupyter/docker-stacks/issues/860