[MESOS-4996] 'containerizer->update' will always fail after killing a docker container.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.0, 0.23.1, 0.24.0, 0.24.1, 0.25.0, 0.26.0, 0.27.0, 0.27.1, 0.27.2, 0.28.0, 1.1.3, 1.2.3, 1.3.1, 1.4.1, 1.5.0
• Fix Version/s: 1.10.0
• Component/s: containerization, docker
• Labels: mesosphere
• Assignee: Unassigned
• Reporter: Jie Yu
• Created: 21/Mar/16 23:30
• Updated: 05/May/20 08:33
• Resolved: 05/May/20 08:33

Description

Here is the sequence of events:
1) the framework does a killTask
2) killTask is handled by the docker executor
3) the docker executor calls docker->stop
4) docker container terminated
5) docker executor sends TASK_KILLED to the agent
6) since TASK_KILLED is terminal, agent calls containerizer->update()
7) DockerContainerizerProcess::update is called
8) Since pid is known, it tries to get the cgroups associated with the pid
9) Since pid has gone, cgroups::cpu::cgroup(pid) will return Error
10) We got "Failed to determine the cgroup hierarchy where the 'cpu' subsystem is mounted: Failed to read /proc/4390/cgroup: Failed to open file '/proc/4390/cgroup': No such file or directory"
11) containerizer->update fail, agent will call containerizer->destroy

Issue Links

• relates to: MESOS-4827 Destroy Docker container crashes Mesos slave
• relates to: MESOS-4899 Mesos slave crash after killing docker container