[YARN-9850] document or revert change in which DefaultContainerExecutor no longer propagates NM env to containers

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: nodemanager
• Assignee: Unassigned
• Reporter: Steven Rand
• Created: 21/Sep/19 14:25
• Updated: 21/Sep/19 14:25

Description

After https://github.com/apache/hadoop/commit/9d4d30243b0fc9630da51a2c17b543ef671d035c, containers launched by the DefaultContainerExecutor no longer inherit the environment of the NodeManager.

I don't object to the commit (I actually prefer the new behavior), but I do think that it's a notable breaking change, as people may be relying on variables in the NM environment for their containers to behave correctly.

As far as I can tell, we don't currently include this behavior change in the release notes for Hadoop 3, and it's a particularly tricky one to track down, since there's no JIRA ticket for it.

I think that we should at least include this change in the release notes for the 3.0.0 release. Arguably it's worth having the DefaultContainerExecutor set inheritParentEnv to true when it creates its ShellCommandExecutor since that preserves the old behavior and is less surprising to users, but I don't feel strongly either way.