[LIVY-987] NPE when waiting for thrift session to start timeout.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.9.0
• Component/s: Thriftserver
• Assignee: Jianzhen Wu
• Reporter: Jianzhen Wu
• Created: 25/Aug/23 09:16
• Updated: 05/Sep/23 10:01
• Resolved: 05/Sep/23 10:01

Description

 

Livy spends 10 min waiting for the session to start. If it takes more than 10 minutes to start, it will throw a Timeout exception. There is no cause for the timeout exception. When Livy throws e.getCause, NPE occurs.

Livy Code

      Try(Await.result(future, maxSessionWait)) match {
        case Success(session) => session
        case Failure(e) => throw e.getCause
      } 

Error Log

23/08/25 16:01:41 INFO  LivyExecuteStatementOperation: (Error executing query, currentState RUNNING, ,java.lang.NullPointerException)
23/08/25 16:01:41 ERROR  LivyExecuteStatementOperation: Error running hive query:
org.apache.hive.service.cli.HiveSQLException: java.lang.NullPointerException
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation.execute(LivyExecuteStatementOperation.scala:186)
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$2$$anon$3.run(LivyExecuteStatementOperation.scala:105)
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$2$$anon$3.run(LivyExecuteStatementOperation.scala:102)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:2038)
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation$$anon$2.run(LivyExecuteStatementOperation.scala:115)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
        at org.apache.livy.thriftserver.LivyThriftSessionManager.getLivySession(LivyThriftSessionManager.scala:99)
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation.rpcClient$lzycompute(LivyExecuteStatementOperation.scala:65)
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation.rpcClient(LivyExecuteStatementOperation.scala:58)
        at org.apache.livy.thriftserver.LivyExecuteStatementOperation.execute(LivyExecuteStatementOperation.scala:173)