Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-617

Livy session leak on Yarn when creating session duplicated names

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.0
    • Fix Version/s: 0.7.0
    • Component/s: Server
    • Labels:
      None

      Description

      When running Livy on Yarn and try to create session with duplicated names, Livy server sends response to client "Duplicate session name: xxx" but it doesn't stop the session. The session creation failed, however, the Yarn application got started and keeps running forever.

      This is because during livy session register method, exception "IllegalArgumentException" is thrown without stopping the session:

      def register(session: S): S = {
          info(s"Registering new session ${session.id}")
          synchronized {
            session.name.foreach { sessionName =>
              if (sessionsByName.contains(sessionName)) {
                throw new IllegalArgumentException(s"Duplicate session name: ${session.name}")
              } else {
                sessionsByName.put(sessionName, session)
              }
            }
            sessions.put(session.id, session)
            session.start()
          }
          session
        }

       

      Reproduction scripts:

      curl -s -k -u username:password -X POST --data '{"name": "duplicatedname", "kind": "pyspark"}' -H "Content-Type: application/json" 'https://myserver/livy/v1/sessions'

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                shanyu shanyu zhao
                Reporter:
                shanyu shanyu zhao
              • Votes:
                1 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 2h 40m
                  2h 40m