Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-515

Livy session is idle even if spark context is unavailable

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.4.0
    • 0.9.0
    • Core
    • None

    Description

      When using livy in the session mode, Livy is not able to figure out if the application master is available.

      Say suppose livy was in the idle state, now for some reason the application master died. Ideally, even the session associated with this application master should either be terminated or should have handled this situation. Instead, the session continues to remain in the Idle state. This is wrong.

       

      Steps to reproduce.

      1. Bring up livy in session mode(I am using 5 sessions).
      2. Kill the application master using "yarn application -kill <application id>"
      3. Now list the application in yarn "yarn application  -list" you will see 4 application master(I had 5, Killed 1 so 4)
      4. Now reload livy UI. You will still see 5 sessions in idle.

      Attached the session logs whose application master was killed.

      Attached are some images depicting the issue

      Attachments

        1. livyUI.png
          38 kB
          Raghavendra
        2. livy-log
          9 kB
          Raghavendra
        3. command.history
          5 kB
          Jie Wang
        4. BeforeKillingLivy.png
          78 kB
          Raghavendra
        5. AfterKillingLivy.png
          51 kB
          Raghavendra

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            raghavendra Raghavendra

            Dates

              Created:
              Updated:

              Slack

                Issue deployment