[MESOS-8125] Agent should properly handle recovering an executor when its pid is reused

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.4.2, 1.5.1, 1.6.0
• Component/s: docker
• Target Version/s: 1.6.0
• Sprint: Mesosphere Sprint 73
• Story Points: 2
• Assignee: Qian Zhang
• Reporter: Gastón Kleiman
• Shepherd: Vinod Kone
• Created: 23/Oct/17 23:44
• Updated: 22/Mar/19 20:20
• Resolved: 31/Jan/18 01:39

Description

Here's how to reproduce this issue:

1. Start a task using the Docker containerizer (the same will probably happen with the command executor).
2. Stop the corresponding Mesos agent while the task is running.
3. Change the executor's checkpointed forked pid, which is located in the meta directory, e.g., /var/lib/mesos/slave/meta/slaves/latest/frameworks/19faf6e0-3917-48ab-8b8e-97ec4f9ed41e-0001/executors/foo.13faee90-b5f0-11e7-8032-e607d2b4348c/runs/latest/pids/forked.pid. I used pid 2, which is normally used by kthreadd.
4. Reboot the host

Issue Links

• is related to: MESOS-6223 Allow agents to re-register post a host reboot
• relates to: MESOS-9501 Mesos executor fails to terminate and gets stuck after agent host reboot.
• relates to: MESOS-9672 Docker containerizer should ignore pids of executors that do not pass the connection check.