[LIVY-874] App in Accepted state fails after Livy restart

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 0.7.0
• Fix Version/s: 0.9.0
• Component/s: RSC
• Assignee: Unassigned
• Reporter: Shrikant Prasad
• Created: 18/Nov/21 10:34
• Updated: 12/Nov/22 22:10

Description

Application in ACCEPTED state fails if there is restart of livy server. Ideally it should move to running state when the resource is available after restart.

The yarn application log has below exception:

Caused by: java.util.concurrent.ExecutionException: io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: **/**:10001
at io.netty.util.concurrent.DefaultPromise.get(DefaultPromise.java:349)
at org.apache.livy.rsc.driver.RSCDriver.initializeServer(RSCDriver.java:203)
at org.apache.livy.rsc.driver.RSCDriver.run(RSCDriver.java:336)
at org.apache.livy.rsc.driver.RSCDriverBootstrapper.main(RSCDriverBootstrapper.java:93)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.spark.deploy.yarn.ApplicationMaster$$anon$2.run(ApplicationMaster.scala:728)

 

If there is no restart of livy server, the app in accepted state successfully moves to running state when resource is freed up.