[YARN-6434] When setting environment variables, can't use comma for a list of value in key = value pairs.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Won't Fix
• Assignee: Unassigned
• Reporter: Jaeboo Jeong
• Created: 04/Apr/17 03:06
• Updated: 13/Apr/18 01:00
• Resolved: 13/Apr/18 00:41

Description

We can set environment variables using yarn.app.mapreduce.am.env, mapreduce.map.env, mapreduce.reduce.env.
There is no problem if we use key=value pairs like X=Y, X=$Y.
However If we want to set key=a list of value pair(e.g. X=Y,Z), we can’t.
This is related to YARN-4595.

The attached patch is based on YARN-3768.
We can set environment variables like below.

mapreduce.map.env="YARN_CONTAINER_RUNTIME_TYPE=docker,YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=hadoop-docker,YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS=\"/dir1:/targetdir1,/dir2:/targetdir2\""

Issue Links

• is related to: YARN-8029 YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS should not use commas as separators
• relates to: MAPREDUCE-7069 Add ability to specify user environment variables individually