[SPARK-17814] spark submit arguments are truncated in yarn-cluster mode

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Won't Fix
• Affects Version/s: 1.6.1
• Component/s: Spark Core, Spark Submit, YARN
• Assignee: Unassigned
• Reporter: shreyas subramanya
• Created: 06/Oct/16 22:34
• Updated: 17/May/20 18:14
• Resolved: 12/Oct/16 07:58

Description

One of our customers is trying to pass in json through spark-submit as follows:
spark-submit --verbose --class SimpleClass --master yarn-cluster ./simple.jar "{\"mode\":\"wf\", \"arrays\":{\"array\":[1]}}"
The application reports the passed arguments as: {"mode":"wf", "arrays":{"array":[1]

If the same application is submitted in yarn-client mode, as follows:
spark-submit --verbose --class SimpleClass --master yarn-client ./simple.jar "{\"mode\":\"wf\", \"arrays\":{\"array\":[1]}}"
The application reports the passed args as: {"mode":"wf", "arrays":{"array":[1]}}

Issue Links

• is duplicated by: OOZIE-2716 Oozie Truncates Workflow Parameter Values which contain '}}'