Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7069

Add ability to specify user environment variables individually

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.2.0, 2.10.1
    • None
    • None
    • Reviewed
    • Hide
      Environment variables for MapReduce tasks can now be specified as separate properties, e.g.:
      mapreduce.map.env.VARNAME=value
      mapreduce.reduce.env.VARNAME=value
      yarn.app.mapreduce.am.env.VARNAME=value
      yarn.app.mapreduce.am.admin.user.env.VARNAME=value
      This form of specifying environment variables is useful when the value of an environment variable contains commas.
      Show
      Environment variables for MapReduce tasks can now be specified as separate properties, e.g.: mapreduce.map.env.VARNAME=value mapreduce.reduce.env.VARNAME=value yarn.app.mapreduce.am.env.VARNAME=value yarn.app.mapreduce.am.admin.user.env.VARNAME=value This form of specifying environment variables is useful when the value of an environment variable contains commas.

    Description

      As reported inĀ YARN-6830, it is currently not possible to specify an environment variable that contains commas via mapreduce.map.env, mapreduce.reduce.env, or mapreduce.admin.user.env.

      To address this, aw proposed in YARN-6830 that we add the ability to specify environment variables individually:

      e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar

      Attachments

        1. MAPREDUCE-7069.001.patch
          19 kB
          Jim Brennan
        2. MAPREDUCE-7069.002.patch
          19 kB
          Jim Brennan
        3. MAPREDUCE-7069.003.patch
          36 kB
          Jim Brennan
        4. MAPREDUCE-7069.004.patch
          37 kB
          Jim Brennan
        5. MAPREDUCE-7069.005.patch
          37 kB
          Jim Brennan
        6. MAPREDUCE-7069.006.patch
          35 kB
          Jim Brennan
        7. MAPREDUCE-7069.007.patch
          35 kB
          Jim Brennan

        Issue Links

          Activity

            People

              jbrennan Jim Brennan
              jbrennan Jim Brennan
              Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: