[MESOS-8981] When Docker daemon restarts, Mesos slave does not know about this container

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Assignee: Unassigned
• Reporter: Lenfree Yeung
• Created: 06/Jun/18 06:31
• Updated: 07/Jun/18 03:31

Description

Mesos version: 1.0.0

Containerizer: Docker

Docker version: 17.09

OS: Ubuntu 16.04

Framework: Singularity

 

Recently I had to roll out a change on Docker daemon config which involves restarting of Docker daemon.

Logs:

 

Executor for container '6d747a2c-3a52-4af6-b472-45e7603c0978' has exited
Destroying container '6d747a2c-3a52-4af6-b472-45e7603c0978'
Running docker stop on container '6d747a2c-3a52-4af6-b472-45e7603c0978'

 
To reproduce:

1. Run an existing job
2. Restart Docker daemon on this host where job is assigned to
3. Mesos  would picked this up and assign this job to another slave which is expected behaviour.
4. Run docker ps will show container running on host from step 1 but Mesos/Scheduler only knows about the new job that was moved to new host. 

Am I missing some configuration that address this? Mesos slave is running with `--containerizers=docker,mesos`

 

Issue Links

• relates to: MESOS-8509 Launching a Docker container with `--restart=always` may cause the Docker container is running after the task completes