Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4310

Shell interpreter fails to execute on new installation of Zeppelin

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.8.1
    • Fix Version/s: None
    • Component/s: build, Interpreters
    • Labels:
      None

      Description

      I just downloaded and installed the netinst binary package of zeppelin 0.8.1 and installed the shell interpreter with the command:

      ./bin/install-interpreter.sh --name shell

       

      If I open up a new note and try to run a shell command

      %sh
      
      ls -lh
      

       

      I get the following error.

      INFO [2019-08-21 13:27:05,832] ({qtp1582797472-104} RemoteInterpreter.java[call]:168) - Create RemoteInterpreter org.apache.zep
      pelin.shell.ShellInterpreter
       INFO [2019-08-21 13:27:05,928] ({qtp1582797472-104} NotebookServer.java[completion]:1413) - Fail to get completion
      java.lang.RuntimeException: org.apache.thrift.TApplicationException: Internal error processing createInterpreter
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:139)
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:165)
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:132)
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.completion(RemoteInterpreter.java:349)
              at org.apache.zeppelin.notebook.Paragraph.completion(Paragraph.java:270)
              at org.apache.zeppelin.notebook.Note.completion(Note.java:729)
              at org.apache.zeppelin.socket.NotebookServer.completion(NotebookServer.java:1411)
              at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:304)
              at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:59)
              at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:189)
              at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
              at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:66)
              at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:158)
              at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:162)
              at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:459)
              at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:182)
              at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:105)
              at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:142)
              at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:85)
              at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:220)
              at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:219)
              at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:244)
              at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:559)
              at org.eclipse.jetty.websocket.common.io.AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:390)
              at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
              at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:103)
              at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
              at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
              at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:765)
              at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:683)
              at java.lang.Thread.run(Thread.java:745)

       

      In the shell interpreter logs it translates to a ClassCastException

      ERROR [2019-08-21 13:27:05,926] ({pool-1-thread-5} ProcessFunction.java[process]:41) - Internal error processing createInterpret
      er
      org.apache.thrift.TException: java.lang.ClassNotFoundException: org.apache.zeppelin.shell.ShellInterpreter
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:320)
              at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpret
      erService.java:1799)
              at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$createInterpreter.getResult(RemoteInterpret
      erService.java:1784)
              at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
              at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
              at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.ClassNotFoundException: org.apache.zeppelin.shell.ShellInterpreter
              at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
              at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
              at java.security.AccessController.doPrivileged(Native Method)
              at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
              at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
              at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
              at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
              at java.lang.Class.forName0(Native Method)
              at java.lang.Class.forName(Class.java:260)
              at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.createInterpreter(RemoteInterpreterServer.java:302)
              ... 8 more

       

      After a little googling I found out that the issue is that zeppelin looks for a specific path when attempting to instantiate interpreters. However, when the shell interpreter is installed using the `install-interpreter.sh` script, it installs it to the `interpreter/shell` folder but the plugin system looks for the classes in the `interpreter/sh` folder.

      A simple workaround for this issue is to rename the shell folder (`mv interpreter/shell interpreter/sh`) and restart the zeppelin daemon. But it would be good to fix the plugin system to look for the shell classes in the `interpreter/shell` directory instead of having manually rename the folder.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              tojo Tim Ojo
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: