[MESOS-2020] mesos should send docker failure messages to scheduler

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.20.1
• Fix Version/s: 0.23.0
• Component/s: agent, docker

Environment

mesos-slaves running in docker

• Assignee: Jay Buffington
• Reporter: Ivan Babrou
• Created: 31/Oct/14 09:04
• Updated: 14/May/15 07:50
• Resolved: 14/May/15 00:05

Description

I tried to start container that cannot actually start for some reason, like this (from slave logs):

{"log":"I1031 08:58:02.692958 60 slave.cpp:1112] Launching task topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799 for framework 20141003-172543-3892422848-5050-1-0000\n","stream":"stderr","time":"2014-10-31T08:58:02.692986684Z"} {"log":"I1031 08:58:02.693696 60 slave.cpp:1222] Queuing task 'topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799' for executor topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799 of framework '20141003-172543-3892422848-5050-1-0000\n","stream":"stderr","time":"2014-10-31T08:58:02.693734418Z"} {"log":"I1031 08:58:02.695612 64 docker.cpp:743] Starting container '02a786c9-556c-4ed9-80d2-1850a49030fe' for task 'topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799' (and executor 'topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799') of framework '20141003-172543-3892422848-5050-1-0000'\n","stream":"stderr","time":"2014-10-31T08:58:02.6956675Z"} {"log":"E1031 08:58:05.272902 65 slave.cpp:2485] Container '02a786c9-556c-4ed9-80d2-1850a49030fe' for executor 'topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799' of framework '20141003-172543-3892422848-5050-1-0000' failed to start: Failed to 'docker run -d -c 204 -m 67108864 -e PORT=31459 -e PORT0=31459 -e PORTS=31459 -e HOST=web544 -e MESOS_SANDBOX=/mnt/mesos/sandbox -v /var/lib/mesos/slave/slaves/20141028-073834-3909200064-5050-1-75/frameworks/20141003-172543-3892422848-5050-1-0000/executors/topface_demo.055a8f8f-60dc-11e4-b4ad-56847afe9799/runs/02a786c9-556c-4ed9-80d2-1850a49030fe:/mnt/mesos/sandbox --net host --name mesos-02a786c9-556c-4ed9-80d2-1850a49030fe docker.core.tf/demo:1': exit status = exited with status 1 stderr = 2014/10/31 08:58:04 Error response from daemon: Cannot start container 05666763bff98ad70f35968add3923018a9d457a1f4e9dab0981936841093d2f: exec: \"/server.js\": permission denied\n","stream":"stderr","time":"2014-10-31T08:58:05.273016253Z"}

But when I go to task sandbox from mesos ui, stdout and stderr are empty.

Marathon keeps scheduling tasks, but they all silently fail, this is very misleading.

Issue Links

• relates to: MESOS-2035 Add reason to containerizer proto Termination