[ZEPPELIN-2564] LivySparkSQLInterpreter throws NullPointerException when getProgress is called

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.7.2, 0.8.0
• Component/s: livy-interpreter
• Assignee: Sergey Mazin
• Reporter: Sergey Mazin
• Created: 17/May/17 15:55
• Updated: 18/May/17 16:13
• Resolved: 18/May/17 14:37

Description

When using %livy.sql interpreter following error is logged into interpreter log:

ERROR [2017-05-17 15:01:32,897] (

{pool-1-thread-1}

TThreadPoolServer.java[run]:296) - Error occurred during processing of message.
java.lang.NullPointerException
at org.apache.zeppelin.livy.BaseLivyInterpreter.getProgress(BaseLivyInterpreter.java:178)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(LazyOpenInterpreter.java:121)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer.getProgress(RemoteInterpreterServer.java:558)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$getProgress.getResult(RemoteInterpreterService.java:1899)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Processor$getProgress.getResult(RemoteInterpreterService.java:1884)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)

and zeppelin server log:
ERROR [2017-05-17 15:01:44,004] (

{JobProgressPoller, jobId=20170511-200435_1144682471}

JobProgressPoller.java[run]:58) - Can not get or update progress
org.apache.zeppelin.interpreter.InterpreterException: org.apache.thrift.transport.TTransportException
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(RemoteInterpreter.java:468)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.getProgress(LazyOpenInterpreter.java:121)
at org.apache.zeppelin.notebook.Paragraph.progress(Paragraph.java:334)
at org.apache.zeppelin.scheduler.JobProgressPoller.run(JobProgressPoller.java:55)
Caused by: org.apache.thrift.transport.TTransportException
at org.apache.thrift.transport.TIOStreamTransport.read(TIOStreamTransport.java:132)
at org.apache.thrift.transport.TTransport.readAll(TTransport.java:86)
at org.apache.thrift.protocol.TBinaryProtocol.readAll(TBinaryProtocol.java:429)
at org.apache.thrift.protocol.TBinaryProtocol.readI32(TBinaryProtocol.java:318)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:219)
at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:69)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.recv_getProgress(RemoteInterpreterService.java:321)
at org.apache.zeppelin.interpreter.thrift.RemoteInterpreterService$Client.getProgress(RemoteInterpreterService.java:306)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getProgress(RemoteInterpreter.java:465)
... 3 more

This happens because this.livyVersion is never assigned in LivySparkSQLInterpreter: https://github.com/apache/zeppelin/blob/master/livy/src/main/java/org/apache/zeppelin/livy/LivySparkSQLInterpreter.java#L63

Issue Links

• links to: GitHub Pull Request #2350