[SPARK-27015] spark-submit does not properly escape arguments sent to Mesos dispatcher

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3.3, 2.4.0
• Fix Version/s: 3.0.0
• Component/s: Mesos
• Assignee: Martin Loncaric
• Reporter: Martin Loncaric
• Created: 28/Feb/19 20:31
• Updated: 06/Mar/19 00:38
• Resolved: 05/Mar/19 21:06

Description

Arguments sent to the dispatcher must be escaped; for instance,

spark-submit --master mesos://url:port my.jar --arg1 "a b$c"

fails, and instead must be submitted as

spark-submit --master mesos://url:port my.jar --arg1 "a\\ b\\$c"

Issue Links

• links to: GitHub Pull Request #23967