Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-7226

Whitelisted variables do not support delayed variable expansion

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.9.0, 2.8.1, 3.0.0-alpha4
    • Fix Version/s: 2.9.0, 2.8.3, 3.0.0
    • Component/s: nodemanager
    • Labels:
      None

      Description

      The nodemanager supports a configurable list of environment variables, via yarn.nodemanager.env-whitelist, that will be propagated to the container's environment unless those variables were specified in the container launch context. Unfortunately the handling of these whitelisted variables prevents using delayed variable expansion. For example, if a user shipped their own version of hadoop with their job via the distributed cache and specified:

      HADOOP_COMMON_HOME={{PWD}}/my-private-hadoop/
      

      as part of their job, the variable will be set as the literal string:

      $PWD/my-private-hadoop/
      

      rather than having $PWD expand to the container's current directory as it does for any other, non-whitelisted variable being set to the same value.

        Attachments

        1. YARN-7226-branch-2.8.006.patch
          25 kB
          Jason Lowe
        2. YARN-7226-branch-2.007.patch
          24 kB
          Jason Lowe
        3. YARN-7226-branch-2.006.patch
          24 kB
          Jason Lowe
        4. YARN-7226.006.patch
          20 kB
          Jason Lowe
        5. YARN-7226.005.patch
          20 kB
          Jason Lowe
        6. YARN-7226.004.patch
          15 kB
          Jason Lowe
        7. YARN-7226.003.patch
          15 kB
          Jason Lowe
        8. YARN-7226.002.patch
          6 kB
          Jason Lowe
        9. YARN-7226.001.patch
          5 kB
          Jason Lowe

          Activity

            People

            • Assignee:
              jlowe Jason Lowe
              Reporter:
              jlowe Jason Lowe
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: