Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-412

CreateSession should be reject when too many child process "spark-submit" is running

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.4.0
    • 0.5.0
    • Server
    • None

    Description

      In our cluster, livy server run with spark yarn cluster mode, when createSession request is too frequently, livyServer will start too more spark-submit child process, it will cause the machine oom. I think livy server should reject the create session request when there is too more spark-submit child process. I have fix it in our own cluster.

      Attachments

        Activity

          People

            shenhong shenh062326
            shenhong shenh062326
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: