We have checked various cases for this issue in detail and had some offline discussions with Varun Vasudev and Bibin A Chundatt also.
Now coming to the issue mentioned.
MAPREDUCE-6704 has come in to a common consensus that HADOOP_MAPRED_HOME could be published with nm-white-list so that even if this variable is not present in env, we can take the configured value. This will solve the container launch problem mentioned there.
As of today, if we publish HADOOP_MAPRED_HOME through nm-whitelist-env, it will look like below in launch_container.sh.
As mentioned above, this will be fine in case of normal container launch. But in docker scenario, the path value given may not be valid. In such case, we will see the failure as mentioned in this jira. Ideally if this environment variable were published as whitelist env, then it could try to take from env itself first.
Here container will try to get the value from HADOOP_MAPRED_HOME if its available in env first. If not, it will take value which is given as "/home/hadoopbuild". In case of docker, this value was set in its system env itself (i think Bibin used sequence iq docker image here which had this value).
So in conclusion, all variables mentioned in yarn.nodemanager.env-whitelist could be whitelisted. This can solve the problem. If some docker images doesn't have this env variable set in its env, and if the default path to HADOOP_MAPRED_HOME is also invalid, then user has to set correct mapred home path in yarn.app.mapreduce.am.env.
YARN-5877.0002.patch is good one to go. Bibin A Chundatt could you rebase the patch and give a latest version number to compile against trunk. If possible, could you please share test results with this latest patch too in docker.