Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-11759

Spark task on mesos with docker fails with sh: 1: /opt/spark/bin/spark-class: not found

    XMLWordPrintableJSON

    Details

    • Type: Question
    • Status: Closed
    • Priority: Major
    • Resolution: Later
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Deploy, Mesos
    • Labels:
      None

      Description

      I'm using Spark 1.5.1 and Mesos 0.25 in cluster mode. I've the spark-dispatcher running, and run spark-submit. The driver is launched, but it fails because it seems that the task it launches fails.
      In the logs of the launched task I can see the following error:
      sh: 1: /opt/spark/bin/spark-class: not found

      I checked my docker image and the /opt/spark/bin/spark-class exists. I then noticed that it's using sh, therefore I tried to run (in the docker image) the following:
      sh /opt/spark/bin/spark-class org.apache.spark.deploy.master.Master

      It fails with the following error:
      spark-class: 73: spark-class: Syntax error: "(" unexpected

      Is this an error in Spark?

      Thanks

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                lmtjalves Luis Alves
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: