[LIVY-339] Unable to start spark session if spark.jars.packages are set in spark config.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.4.0
• Fix Version/s: 0.9.0
• Component/s: Core, REPL
• Environment: Spark 2.1 running on Hadoop 2.8
• Assignee: Unassigned
• Reporter: Chetan Kumar Bhatt
• Created: 10/Apr/17 16:49
• Updated: 12/Nov/22 22:09

Description

I have added following configuration to spark-defaults.conf

 spark.jars.packages com.amazonaws:aws-java-sdk:1.11.115,org.apache.hadoop:hadoop-aws:2.8.0

If I start spark-shell with this configuration, spark-shell loads these jars and starts without any issue.

But when I use Jupyter with Livy, I am getting error as shown in attached error.png.
Same Jupyter+Livy+Spark combo works well with default spark configuration as shown in screenshot success.png