Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1379

Flink interpreter missing scala libraries

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.1
    • Fix Version/s: 0.6.2, 0.7.0
    • Component/s: zeppelin-interpreter
    • Labels:
      None

      Description

      Original post : https://www.mail-archive.com/users@zeppelin.apache.org/msg00861.html

      I tried the latest zeppelin release (binary with all interpreters), and in order to make the flink interpreter work in non-local mode, i had to copy all scala libraries from another interpreter to the flink interpreter.

      After trying to connect to an existing flink cluster i got

      zeppelin_1 | Exception in thread "pool-1-thread-2" java.lang.NoClassDefFoundError: scala/collection/Seq
      zeppelin_1 | at java.lang.Class.forName0(Native Method)
      zeppelin_1 | at java.lang.Class.forName(Class.java:264)
      zeppelin_1 | at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:148)
      zeppelin_1 | at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1409)
      zeppelin_1 | at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpreterService.java:1394)
      zeppelin_1 | at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
      zeppelin_1 | at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
      zeppelin_1 | at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
      zeppelin_1 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      zeppelin_1 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      zeppelin_1 | at java.lang.Thread.run(Thread.java:745)
      zeppelin_1 | Caused by: java.lang.ClassNotFoundException: scala.collection.Seq
      zeppelin_1 | at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      zeppelin_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      zeppelin_1 | at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
      zeppelin_1 | at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      zeppelin_1 | ... 11 more

      to get it working i did

      RUN cp /opt/zeppelin/interpreter/ignite/scala*jar /opt/zeppelin/interpreter/flink/

      which would copy:

      scala-compiler-2.11.7.jar
      scala-library-2.11.7.jar
      scala-parser-combinators_2.11-1.0.4.jar
      scala-reflect-2.11.7.jar
      scala-xml_2.11-1.0.4.jar

      According to Trevor Grant (who helped me out on flink-user), that is a regression of upgrading Zeppelin to spark 2.0/Scala 2.11.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                lresende Luciano Resende
                Reporter:
                lresende Luciano Resende
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: