[LIVY-755] Livy Batch job Skiping remote jar from hdfs:

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.4.0
• Fix Version/s: 0.9.0
• Component/s: Batch
• Assignee: Unassigned
• Reporter: Chris aby jose
• Created: 02/Apr/20 23:27
• Updated: 12/Nov/22 22:09

Description

I am trying to submit a Livy batch job.But it gives the following error. Can you please help.

Looks like the jar is not getting included while creating Livy session.I tried loading the jar from local as well as hdfs.

For trying from local I updated config to add livy.file.local-dir-whitelist and tried with url as "local:/path/to/file*

I verifed the files is present in HDFS on the correct location.

 
"Warning: Skip remote jar hdfs://QuantumHadoopDev/livy_qunatum/SampleSparkProject-0.0.2-SNAPSHOT.jar.",
  "java.lang.ClassNotFoundException: App",
  " at java.net.URLClassLoader.findClass(URLClassLoader.java:381)",
  " at java.lang.ClassLoader.loadClass(ClassLoader.java:424)",
  " at java.lang.ClassLoader.loadClass(ClassLoader.java:357)",
  " at java.lang.Class.forName0(Native Method)",
  " at java.lang.Class.forName(Class.java:348)",
  " at org.apache.spark.util.Utils$.classForName(Utils.scala:175