[MESOS-9231] `docker inspect` may return an unexpected result to Docker executor due to a race condition.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.4.2, 1.5.1, 1.6.1, 1.7.0
• Fix Version/s: 1.4.3, 1.5.2, 1.6.2, 1.7.1, 1.8.0
• Sprint: Mesosphere Sprint 2018-29, Mesosphere RI-6 Sprint 2018-30
• Story Points: 3
• Assignee: Qian Zhang
• Reporter: Qian Zhang
• Shepherd: Gilbert Song
• Created: 13/Sep/18 23:35
• Updated: 16/Jan/19 00:42
• Resolved: 30/Sep/18 07:06

Description

In the Docker container (`src/docker/executor`), we call `docker inspect` right after `docker run` (https://github.com/apache/mesos/blob/1.6.0/src/docker/executor.cpp#L230:L242), there is a small chance for `docker inspect` to return an unexpected result which does not contain the Docker container ID, so we will see an error like below:

E0830 00:09:37.303499 2428 executor.cpp:385] Failed to inspect container 'mesos-eaa4f455-0a2c-47ff-bf98-8bd0ad243740': Unable to create container: Unable to find Id in container

If that happens, Docker executor will not send `TASK_RUNNING` status update, so the task will be stuck at `TASK_STARTING`.