[LIVY-636] Unable to create interactive session with additional JAR in spark.driver.extraClassPath

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Fix Version/s: 0.9.0
• Assignee: Unassigned
• Reporter: Ishita Virmani
• Created: 13/Aug/19 11:44
• Updated: 12/Nov/22 22:09

Description

Command Run: c{{url -H "Content-Type: application/json" -X POST -d '{"kind":"pyspark","conf":{"spark.driver.extraClassPath":"/data/XXX-0.0.1-SNAPSHOT.jar"}}' -i http://<LIVY_SERVER_IP:PORT>/session}}

The above command fails to create a Spark Session on YARN with Null pointer exception. Stack trace for the same has been attached along-with.

The JAR file here is present on local driver Path. Also tried using HDFS path in the following manner hdfs://<NM_IP>:<NM_Port>/data/XXX-0.0.1-SNAPSHOT.jar