[LIVY-515] Livy session is idle even if spark context is unavailable

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.4.0
• Fix Version/s: 0.9.0
• Component/s: Core
• Assignee: Unassigned
• Reporter: Raghavendra
• Created: 17/Sep/18 10:43
• Updated: 12/Nov/22 22:28

Description

When using livy in the session mode, Livy is not able to figure out if the application master is available.

Say suppose livy was in the idle state, now for some reason the application master died. Ideally, even the session associated with this application master should either be terminated or should have handled this situation. Instead, the session continues to remain in the Idle state. This is wrong.

 

Steps to reproduce.

1. Bring up livy in session mode(I am using 5 sessions).
2. Kill the application master using "yarn application -kill <application id>"
3. Now list the application in yarn "yarn application  -list" you will see 4 application master(I had 5, Killed 1 so 4)
4. Now reload livy UI. You will still see 5 sessions in idle.

Attached the session logs whose application master was killed.

Attached are some images depicting the issue