Uploaded image for project: 'Livy'
  1. Livy
  2. LIVY-246

Support multiple Spark home in runtime

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 0.3
    • 0.9.0
    • Server
    • None

    Description

      Current SPARK_HOME is a global configuration for LivyServer, it will check the spark home as well as spark version using this configuration when LivyServer is started. This limits the usage scenario of Livy. Since current Livy could support different Spark in one assembly, but switching to different Spark still requires stopping LivyServer, modify SPARK_HOME and restart. It would be better to support different Sparks in the runtime with session based, so user could switch different Sparks without needing to stop LivyServer, also it is useful for user who wants to operate with different Sparks.

      So here propose to move SPARK_HOME from global configuration to session based configuration.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jerryshao Saisai Shao
            jerryshao Saisai Shao

            Dates

              Created:
              Updated:

              Slack

                Issue deployment