[SPARK-11759] Spark task on mesos with docker fails with sh: 1: /opt/spark/bin/spark-class: not found

• Type: Question
• Status: Closed
• Priority: Major
• Resolution: Later
• Component/s: Deploy, Mesos
• Assignee: Unassigned
• Reporter: Luis Alves
• Created: 16/Nov/15 15:24
• Updated: 04/Feb/16 17:21
• Resolved: 04/Feb/16 17:21

Description

I'm using Spark 1.5.1 and Mesos 0.25 in cluster mode. I've the spark-dispatcher running, and run spark-submit. The driver is launched, but it fails because it seems that the task it launches fails.
In the logs of the launched task I can see the following error:
sh: 1: /opt/spark/bin/spark-class: not found

I checked my docker image and the /opt/spark/bin/spark-class exists. I then noticed that it's using sh, therefore I tried to run (in the docker image) the following:
sh /opt/spark/bin/spark-class org.apache.spark.deploy.master.Master

It fails with the following error:
spark-class: 73: spark-class: Syntax error: "(" unexpected

Is this an error in Spark?

Thanks

Issue Links

• relates to: SPARK-12345 Mesos cluster mode is broken