Description
Trying to run on Yarn (hadoop 0.23) I get the following exception:
Exception in thread "main" java.lang.NoSuchFieldException: DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH
at java.lang.Class.getField(Class.java:1569)
at org.apache.spark.deploy.yarn.ClientBase$.getDefaultMRApplicationClasspath(ClientBase.scala:417)
at org.apache.spark.deploy.yarn.ClientBase$$anonfun$6.apply(ClientBase.scala:393)
at org.apache.spark.deploy.yarn.ClientBase$$anonfun$6.apply(ClientBase.scala:393)
at scala.Option.getOrElse(Option.scala:120)
at org.apache.spark.deploy.yarn.ClientBase$.populateHadoopClasspath(ClientBase.scala:392)
at org.apache.spark.deploy.yarn.ClientBase$.populateClasspath(ClientBase.scala:444)
at org.apache.spark.deploy.yarn.ClientBase$class.setupLaunchEnv(ClientBase.scala:274)
at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:37)
at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:67)
at org.apache.spark.deploy.yarn.Client.run(Client.scala:84)
at org.apache.spark.deploy.yarn.Client$.main(Client.scala:177)
at org.apache.spark.deploy.yarn.Client.main(Client.scala)
Attachments
Issue Links
- relates to
-
SPARK-1064 Make it possible to use cluster's Hadoop jars when running against YARN
- Resolved