Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8257

Native service should automatically adding escapes for environment/launch cmd before sending to YARN

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: yarn-native-services
    • Labels:
      None
    • Target Version/s:

      Description

      Noticed this issue while using native service: 

      Basically, when a string for environment / launch command contains chars like ", /, `: it needs to be escaped twice.

      The first time is from json spec, because of json accept double quote only, it needs an escape.

      The second time is from launch container, what we did for command line is: (ContainerLaunch.java)

      line("exec /bin/bash -c \"", StringUtils.join(" ", command), "\"");

      And for environment:

      line("export ", key, "=\"", value, "\"");

      An example of launch_command: 

      "launch_command": "export CLASSPATH=\\`\\$HADOOP_HDFS_HOME/bin/hadoop classpath --glob\\`"

      And example of environment:

      "TF_CONFIG" : "{\\\"cluster\\\": {\\\"master\\\": [\\\"master-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"ps\\\": [\\\"ps-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"], \\\"worker\\\": [\\\"worker-0.distributed-tf.ambari-qa.tensorflow.site:8000\\\"]}, \\\"task\\\": {\\\"type\\\":\\\"${COMPONENT_NAME}\\\", \\\"index\\\":${COMPONENT_ID}}, \\\"environment\\\":\\\"cloud\\\"}",

      To improve usability, I think we should auto escape the input string once. (For example, if user specified

      "TF_CONFIG": "\"key\""
      

      We will automatically escape it to:

      "TF_CONFIG": \\\"key\\\"
      

        Attachments

          Activity

            People

            • Assignee:
              gsaha Gour Saha
              Reporter:
              leftnoteasy Wangda Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated: