[TOREE-430] ClassNotFoundException with lambdas on CDH 5.10.0

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.1.0
• Component/s: Kernel
• Environment: CDH 5.10.0 (Spark 1.6.0, Scala 2.10.5)
• Assignee: Unassigned
• Reporter: Igor Melnichenko
• Created: 08/Aug/17 16:05
• Updated: 13/Nov/17 04:11

Description

In the specified environment, Toree classloader fails to load classes defined and compiled in REPL. For instance, the following Spark example fails with ClassNotFoundException and works fine if the lambda is removed:

Example.scala

val NUM_SAMPLES = 100
val count = sc.parallelize(1 to NUM_SAMPLES).filter { _ =>
  val x = math.random
  val y = math.random
  x*x + y*y < 1
}.count()
println(s"Pi is roughly ${4.0 * count / NUM_SAMPLES}")

Cause: a location of a directory used by a REPL compiler to store runtime-compiled classes is different from a location where classloader tries to find these classes.

PR with fix will be opened soon.

Issue Links

• links to: GitHub Pull Request #129