[MAPREDUCE-7069] Add ability to specify user environment variables individually

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 3.2.0, 2.10.1
• Hadoop Flags: Reviewed

Release Note

Environment variables for MapReduce tasks can now be specified as separate properties, e.g.:
mapreduce.map.env.VARNAME=value
mapreduce.reduce.env.VARNAME=value
yarn.app.mapreduce.am.env.VARNAME=value
yarn.app.mapreduce.am.admin.user.env.VARNAME=value
This form of specifying environment variables is useful when the value of an environment variable contains commas.

• Assignee: Jim Brennan
• Reporter: Jim Brennan
• Created: 28/Mar/18 15:09
• Updated: 10/Jun/21 08:10
• Resolved: 12/Apr/18 16:17

Description

As reported in YARN-6830, it is currently not possible to specify an environment variable that contains commas via mapreduce.map.env, mapreduce.reduce.env, or mapreduce.admin.user.env.

To address this, aw proposed in YARN-6830 that we add the ability to specify environment variables individually:

e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar

Issue Links

• is related to: YARN-6830 Support quoted strings for environment variables
• is related to: YARN-8071 Add ability to specify nodemanager environment variables individually
• is related to: YARN-6434 When setting environment variables, can't use comma for a list of value in key = value pairs.
• is related to: HADOOP-11640 add user defined delimiter support to Configuration