[YARN-8257] Native service should automatically adding escapes for environment/launch cmd before sending to YARN

• Type: Bug
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: yarn-native-services
• Assignee: Gour Saha
• Reporter: Wangda Tan
• Created: 07/May/18 23:56
• Updated: 10/Jun/21 08:15

Description

Noticed this issue while using native service: 

Basically, when a string for environment / launch command contains chars like ", /, `: it needs to be escaped twice.

The first time is from json spec, because of json accept double quote only, it needs an escape.

The second time is from launch container, what we did for command line is: (ContainerLaunch.java)

line("exec /bin/bash -c \"", StringUtils.join(" ", command), "\"");

And for environment:

line("export ", key, "=\"", value, "\"");

An example of launch_command: 

"launch_command": "export CLASSPATH=\\`\\$HADOOP_HDFS_HOME/bin/hadoop classpath --glob\\`"

And example of environment:

"TF_CONFIG" : "{\\\"cluster\\\": {\\\"master\\\": [\\\"master-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"ps\\\": [\\\"ps-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"worker\\\": [\\\"worker-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"]}, \\\"task\\\": {\\\"type\\\":\\\"${COMPONENT_NAME}\\\", \\\"index\\\":${COMPONENT_ID}}, \\\"environment\\\":\\\"cloud\\\"}",

To improve usability, I think we should auto escape the input string once. (For example, if user specified

"TF_CONFIG": "\"key\""

We will automatically escape it to:

"TF_CONFIG": \\\"key\\\"