Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9850

document or revert change in which DefaultContainerExecutor no longer propagates NM env to containers

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: nodemanager
    • Labels:
      None

      Description

      After https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c, containers launched by the DefaultContainerExecutor no longer inherit the environment of the NodeManager.

      I don't object to the commit (I actually prefer the new behavior), but I do think that it's a notable breaking change, as people may be relying on variables in the NM environment for their containers to behave correctly.

      As far as I can tell, we don't currently include this behavior change in the release notes for Hadoop 3, and it's a particularly tricky one to track down, since there's no JIRA ticket for it.

      I think that we should at least include this change in the release notes for the 3.0.0 release. Arguably it's worth having the DefaultContainerExecutor set inheritParentEnv to true when it creates its ShellCommandExecutor since that preserves the old behavior and is less surprising to users, but I don't feel strongly either way.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              Steven Rand Steven Rand
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: