When a configuration property contains shell characters that require quoting, the Mesos cluster scheduler generates the spark-submit argument like so:
Note the quotes around the property value as well as the key=value pair. When using docker, this breaks the spark-submit command and causes the "|" to be interpreted as an actual shell PIPE. Spaces, semi-colons, etc also cause issues.
Although I haven't tried, I suspect this is also a potential security issue in that someone could exploit it to run arbitrary code on the host.
My patch is pretty minimal and just removes the outer quotes around the key=value pair, resulting in something like:
A more extensive fix might try wrapping the entire key=value pair in single quotes but I was concerned about backwards compatibility with that change.