Description
Current SPARK_HOME is a global configuration for LivyServer, it will check the spark home as well as spark version using this configuration when LivyServer is started. This limits the usage scenario of Livy. Since current Livy could support different Spark in one assembly, but switching to different Spark still requires stopping LivyServer, modify SPARK_HOME and restart. It would be better to support different Sparks in the runtime with session based, so user could switch different Sparks without needing to stop LivyServer, also it is useful for user who wants to operate with different Sparks.
So here propose to move SPARK_HOME from global configuration to session based configuration.
Attachments
Issue Links
- is a child of
-
LIVY-106 Support multiple versions of Spark
-
- Open
-