Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-515

Livy session is idle even if spark context is unavailable

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 0.4.0
    • Fix Version/s: None
    • Component/s: Core
    • Labels:
      None

      Description

      When using livy in the session mode, Livy is not able to figure out if the application master is available.

      Say suppose livy was in the idle state, now for some reason the application master died. Ideally, even the session associated with this application master should either be terminated or should have handled this situation. Instead, the session continues to remain in the Idle state. This is wrong.

       

      Steps to reproduce.

      1. Bring up livy in session mode(I am using 5 sessions).
      2. Kill the application master using "yarn application -kill <application id>"
      3. Now list the application in yarn "yarn application  -list" you will see 4 application master(I had 5, Killed 1 so 4)
      4. Now reload livy UI. You will still see 5 sessions in idle.

      Attached the session logs whose application master was killed.

      Attached are some images depicting the issue

        Attachments

        1. AfterKillingLivy.png
          51 kB
          Raghavendra
        2. BeforeKillingLivy.png
          78 kB
          Raghavendra
        3. command.history
          5 kB
          runzhiwang
        4. livy-log
          9 kB
          Raghavendra
        5. livyUI.png
          38 kB
          Raghavendra

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              raghavendra Raghavendra
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: