Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7069

Add ability to specify user environment variables individually

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Environment variables for MapReduce tasks can now be specified as separate properties, e.g.:
      mapreduce.map.env.VARNAME=value
      mapreduce.reduce.env.VARNAME=value
      yarn.app.mapreduce.am.env.VARNAME=value
      yarn.app.mapreduce.am.admin.user.env.VARNAME=value
      This form of specifying environment variables is useful when the value of an environment variable contains commas.
      Show
      Environment variables for MapReduce tasks can now be specified as separate properties, e.g.: mapreduce.map.env.VARNAME=value mapreduce.reduce.env.VARNAME=value yarn.app.mapreduce.am.env.VARNAME=value yarn.app.mapreduce.am.admin.user.env.VARNAME=value This form of specifying environment variables is useful when the value of an environment variable contains commas.

      Description

      As reported inĀ YARN-6830, it is currently not possible to specify an environment variable that contains commas via mapreduce.map.env, mapreduce.reduce.env, or mapreduce.admin.user.env.

      To address this, Allen Wittenauer proposed in YARN-6830 that we add the ability to specify environment variables individually:

      e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar

        Attachments

        1. MAPREDUCE-7069.001.patch
          19 kB
          Jim Brennan
        2. MAPREDUCE-7069.002.patch
          19 kB
          Jim Brennan
        3. MAPREDUCE-7069.003.patch
          36 kB
          Jim Brennan
        4. MAPREDUCE-7069.004.patch
          37 kB
          Jim Brennan
        5. MAPREDUCE-7069.005.patch
          37 kB
          Jim Brennan
        6. MAPREDUCE-7069.006.patch
          35 kB
          Jim Brennan
        7. MAPREDUCE-7069.007.patch
          35 kB
          Jim Brennan

          Issue Links

            Activity

              People

              • Assignee:
                Jim_Brennan Jim Brennan
                Reporter:
                Jim_Brennan Jim Brennan
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: