[ZEPPELIN-1379] Flink interpreter missing scala libraries

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.1
• Fix Version/s: 0.6.2, 0.7.0
• Component/s: zeppelin-interpreter
• Assignee: Luciano Resende
• Reporter: Luciano Resende
• Created: 26/Aug/16 19:45
• Updated: 05/Sep/16 09:23
• Resolved: 05/Sep/16 09:23

Description

Original post : https://www.mail-archive.com/[e-mail]/msg00861.html

I tried the latest zeppelin release (binary with all interpreters), and in order to make the flink interpreter work in non-local mode, i had to copy all scala libraries from another interpreter to the flink interpreter.

After trying to connect to an existing flink cluster i got

zeppelin_1 | Exception in thread "pool-1-thread-2" java.lang.NoClassDefFoundError: scala/collection/Seq
zeppelin_1 | at java.lang.Class.forName0(Native Method)
zeppelin_1 | at java.lang.Class.forName(Class.java:264)
zeppelin_1 | at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:148)
zeppelin_1 | at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1409)
zeppelin_1 | at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1394)
zeppelin_1 | at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
zeppelin_1 | at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
zeppelin_1 | at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
zeppelin_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
zeppelin_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
zeppelin_1 | at java.lang.Thread.run(Thread.java:745)
zeppelin_1 | Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
zeppelin_1 | at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
zeppelin_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
zeppelin_1 | at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
zeppelin_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
zeppelin_1 | ... 11 more

to get it working i did

RUN cp /opt/zeppelin/interpreter/ignite/scala*jar /opt/zeppelin/interpreter/flink/

which would copy:

scala-compiler-2.11.7.jar
scala-library-2.11.7.jar
scala-parser-combinators_2.11-1.0.4.jar
scala-reflect-2.11.7.jar
scala-xml_2.11-1.0.4.jar

According to Trevor Grant (who helped me out on flink-user), that is a regression of upgrading Zeppelin to spark 2.0/Scala 2.11.

Issue Links

• blocks: ZEPPELIN-1347 Release 0.6.2
• links to: GitHub Pull Request #1370