Major Description
In SparkContextInitializer.spark2CreateContext, livy checks for whether catalog implementation is "hive" or "in-memory" to properly initialize the SparkSession.
If the config is set to "hive" (this is the case if livy.repl.enable-hive-context is set to "true"), livy will further check if hive classes are present or not.
It only invokes the enableHiveSupport method of SparkSession.Builder if hive classes are found.
If not, it proceeds to initializing a SparkSession without Hive support enabled.
However, in Spark's code base, when creating SparkSession, it checks for spark.sql.catalogImplementation to determine which session state to initialize.
If spark.sql.catalogImplementation is set to "hive", it will initialize a HiveSessionState.
However, since Livy has already checked that hive classes are not present, it should not go ahead invoking SparkSession.builder.getOrCreate without overwriting spark.sql.catalogImplementation to "in-memory".