[ZEPPELIN-3530] pyspark interpreter error when using ipython

• Type: Bug
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Affects Version/s: 0.8.1
• Fix Version/s: 0.9.0
• Component/s: pySpark
• Labels: pull-request-available
• Assignee: Unassigned
• Reporter: zhenhuanli708
• Created: 05/Jun/18 13:20
• Updated: 23/Jan/19 02:39

Description

version:branch-0.8

build : mvn clean package -Pspark-2.1 -Dhadoop.version=2.7.0 -Phadoop-2.7 -Pvendor-repo -Psparkr -DskipTests -Pscala-2.11

Set up ipython environment as document:

https://zeppelin.apache.org/docs/0.8.0-SNAPSHOT/interpreter/python.html#ipython-support

pip install jupyter

pip install grpcio

and also 

pip install protobuf

Change conf

zeppelin.pyspark.useIPython true

Then run code:

%spark.pyspark

print(spark)

This code throw an exception:

__java.lang.NoSuchMethodError: io.grpc.protobuf.ProtoUtils.marshaller(Lorg/apache/zeppelin/com/google/protobuf/Message;)Lio/grpc/MethodDescriptor$Marshaller;
at org.apache.zeppelin.python.proto.IPythonGrpc.<clinit>(IPythonGrpc.java:40)
at org.apache.zeppelin.python.IPythonClient.<init>(IPythonClient.java:73)
at org.apache.zeppelin.python.IPythonInterpreter.open(IPythonInterpreter.java:149)
at org.apache.zeppelin.spark.IPySparkInterpreter.open(IPySparkInterpreter.java:64)
at org.apache.zeppelin.spark.PySparkInterpreter.open(PySparkInterpreter.java:129)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:617)
at org.apache.zeppelin.scheduler.Job.run(Job.java:188)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:140)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
 

Issue Links

• links to: GitHub Pull Request #3007