[YARN-6639] Docker container can not find launch_container.sh in workdir

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 3.0.0-alpha2, 3.0.0-alpha4
• Component/s: nodemanager
• Labels: easyfix

Docs Text

Here is my log:
2017-05-24 11:03:09,662 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: private script path: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/nmPrivate/application_1495644587391_0001/container_1495644587391_0001_02_000001/launch_container.sh
2017-05-24 11:03:09,695 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: private token path: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/nmPrivate/application_1495644587391_0001/container_1495644587391_0001_02_000001/container_1495644587391_0001_02_000001.tokens
2017-05-24 11:03:09,837 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: private jar path: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/nmPrivate/application_1495644587391_0001/container_1495644587391_0001_02_000001
2017-05-24 11:03:09,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.ContainerImpl: Container container_1495644587391_0001_02_000001 transitioned from SCHEDULED to RUNNING
2017-05-24 11:03:09,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1495644587391_0001_02_000001
2017-05-24 11:03:09,876 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.linux.runtime.DockerLinuxContainerRuntime: container working dir: /home/cwei/project/hadoop-3.0.0-alpha2/yarn-temp/nm-local-dir/usercache/cwei/appcache/application_1495644587391_0001/container_1495644587391_0001_02_000001


Since ContainerLaunch will write the launch script as well as token file to
/yarn-temp/nm-local-dir/nmPrivate/application_id/container_id.
However, the docker working dir is
/yarn-temp/nm-local-dir/usercache/user/appcache/application_id/container_id. It's a miss match that docker can not locate the launch script.


I found launch script is initially written to a temporary directory, and finally should be copied to workdir. This logic is implemented in DefaultContainerExecutor, but did not find why it is not implemented in LinuxContainerExecutor

• Assignee: Unassigned
• Reporter: Wei Chen
• Created: 24/May/17 17:16
• Updated: 27/Apr/18 12:23
• Resolved: 27/Apr/18 12:23

Description

I am testing with DockerLinuxContainerRuntime. I found applications failed to start during initializing the docker container. That's caused by the docker can not find launch_contaienr.sh in its workdir.