[YARN-6830] Support quoted strings for environment variables

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Assignee: Jim Brennan
• Reporter: Shane Kumpf
• Due: 31/Aug/17
• Created: 17/Jul/17 11:26
• Updated: 12/Apr/18 16:23
• Resolved: 12/Apr/18 16:23

Description

There are cases where it is necessary to allow for quoted string literals within environment variables values when passed via the yarn command line interface.

For example, consider the follow environment variables for a MR map task.

MODE=bar
IMAGE_NAME=foo
MOUNTS=/tmp/foo,/tmp/bar

When running the MR job, these environment variables are supplied as a comma delimited string.

-Dmapreduce.map.env="MODE=bar,IMAGE_NAME=foo,MOUNTS=/tmp/foo,/tmp/bar"

In this case, MOUNTS will be parsed and added to the task environment as MOUNTS=/tmp/foo. Any attempts to quote the embedded comma separated value results in quote characters becoming part of the value, and parsing still breaks down at the comma.

This issue is to allow for quoting the comma separated value (escaped double or single quote). This was mentioned on YARN-4595 and will impact YARN-5534 as well.

Issue Links

• is related to: YARN-8029 YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS should not use commas as separators
• is related to: YARN-8071 Add ability to specify nodemanager environment variables individually
• relates to: MAPREDUCE-7069 Add ability to specify user environment variables individually