Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7226

Whitelisted variables do not support delayed variable expansion

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.9.0, 2.8.1, 3.0.0-alpha4
    • 2.9.0, 2.8.3, 3.0.0
    • nodemanager
    • None

    Description

      The nodemanager supports a configurable list of environment variables, via yarn.nodemanager.env-whitelist, that will be propagated to the container's environment unless those variables were specified in the container launch context. Unfortunately the handling of these whitelisted variables prevents using delayed variable expansion. For example, if a user shipped their own version of hadoop with their job via the distributed cache and specified:

      HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
      

      as part of their job, the variable will be set as the literal string:

      $PWD/my-private-hadoop/
      

      rather than having $PWD expand to the container's current directory as it does for any other, non-whitelisted variable being set to the same value.

      Attachments

        1. YARN-7226.001.patch
          5 kB
          Jason Darrell Lowe
        2. YARN-7226.002.patch
          6 kB
          Jason Darrell Lowe
        3. YARN-7226.003.patch
          15 kB
          Jason Darrell Lowe
        4. YARN-7226.004.patch
          15 kB
          Jason Darrell Lowe
        5. YARN-7226.005.patch
          20 kB
          Jason Darrell Lowe
        6. YARN-7226.006.patch
          20 kB
          Jason Darrell Lowe
        7. YARN-7226-branch-2.006.patch
          24 kB
          Jason Darrell Lowe
        8. YARN-7226-branch-2.8.006.patch
          25 kB
          Jason Darrell Lowe
        9. YARN-7226-branch-2.007.patch
          24 kB
          Jason Darrell Lowe

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jlowe Jason Darrell Lowe Assign to me
            jlowe Jason Darrell Lowe
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Issue deployment