Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-1233

spark on hadoop 0.23 yarn fails to run: java.lang.NoSuchFieldException: DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Blocker
    • Resolution: Fixed
    • 1.0.0
    • 1.0.0
    • YARN
    • None

    Description

      Trying to run on Yarn (hadoop 0.23) I get the following exception:

      Exception in thread "main" java.lang.NoSuchFieldException: DEFAULT_MAPREDUCE_APPLICATION_CLASSPATH
      at java.lang.Class.getField(Class.java:1569)
      at org.apache.spark.deploy.yarn.ClientBase$.getDefaultMRApplicationClasspath(ClientBase.scala:417)
      at org.apache.spark.deploy.yarn.ClientBase$$anonfun$6.apply(ClientBase.scala:393)
      at org.apache.spark.deploy.yarn.ClientBase$$anonfun$6.apply(ClientBase.scala:393)
      at scala.Option.getOrElse(Option.scala:120)
      at org.apache.spark.deploy.yarn.ClientBase$.populateHadoopClasspath(ClientBase.scala:392)
      at org.apache.spark.deploy.yarn.ClientBase$.populateClasspath(ClientBase.scala:444)
      at org.apache.spark.deploy.yarn.ClientBase$class.setupLaunchEnv(ClientBase.scala:274)
      at org.apache.spark.deploy.yarn.Client.setupLaunchEnv(Client.scala:37)
      at org.apache.spark.deploy.yarn.Client.runApp(Client.scala:67)
      at org.apache.spark.deploy.yarn.Client.run(Client.scala:84)
      at org.apache.spark.deploy.yarn.Client$.main(Client.scala:177)
      at org.apache.spark.deploy.yarn.Client.main(Client.scala)

      Attachments

        Issue Links

          Activity

            People

              tgraves Thomas Graves
              tgraves Thomas Graves
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: