Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-7069

Add ability to specify user environment variables individually

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.2.0, 2.10.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Environment variables for MapReduce tasks can now be specified as separate properties, e.g.:
      mapreduce.map.env.VARNAME=value
      mapreduce.reduce.env.VARNAME=value
      yarn.app.mapreduce.am.env.VARNAME=value
      yarn.app.mapreduce.am.admin.user.env.VARNAME=value
      This form of specifying environment variables is useful when the value of an environment variable contains commas.
      Show
      Environment variables for MapReduce tasks can now be specified as separate properties, e.g.: mapreduce.map.env.VARNAME=value mapreduce.reduce.env.VARNAME=value yarn.app.mapreduce.am.env.VARNAME=value yarn.app.mapreduce.am.admin.user.env.VARNAME=value This form of specifying environment variables is useful when the value of an environment variable contains commas.

      Description

      As reported inĀ YARN-6830, it is currently not possible to specify an environment variable that contains commas via mapreduce.map.env, mapreduce.reduce.env, or mapreduce.admin.user.env.

      To address this, Allen Wittenauer proposed in YARN-6830 that we add the ability to specify environment variables individually:

      e.g, mapreduce.map.env.[foo]=bar gets turned into foo=bar

        Attachments

        1. MAPREDUCE-7069.001.patch
          19 kB
          Jim Brennan
        2. MAPREDUCE-7069.002.patch
          19 kB
          Jim Brennan
        3. MAPREDUCE-7069.003.patch
          36 kB
          Jim Brennan
        4. MAPREDUCE-7069.004.patch
          37 kB
          Jim Brennan
        5. MAPREDUCE-7069.005.patch
          37 kB
          Jim Brennan
        6. MAPREDUCE-7069.006.patch
          35 kB
          Jim Brennan
        7. MAPREDUCE-7069.007.patch
          35 kB
          Jim Brennan

        Issue Links

          Activity

            People

            • Assignee:
              jbrennan Jim Brennan
              Reporter:
              jbrennan Jim Brennan

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment