[MESOS-7506] Multiple tests leave orphan containers.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.6.0
• Component/s: containerization
• Labels: containerizer flaky-test mesosphere

Environment

Ubuntu 16.04
Fedora 23
other Linux distros

• Sprint: Mesosphere Sprint 66, Mesosphere Sprint 67, Mesosphere Sprint 68, Mesosphere Sprint 69, Mesosphere Sprint 70, Mesosphere Sprint 72, Mesosphere Sprint 73
• Story Points: 8
• Assignee: Andrei Budnik
• Reporter: Alex R
• Shepherd: Alex R
• Created: 15/May/17 18:58
• Updated: 09/Oct/18 20:37
• Resolved: 25/Jan/18 12:22

Description

I've observed a number of flaky tests that leave orphan containers upon cleanup. A typical log looks like this:

../../src/tests/cluster.cpp:580: Failure
Value of: containers->empty()
  Actual: false
Expected: true
Failed to destroy containers: { da3e8aa8-98e7-4e72-a8fd-5d0bae960014 }

All currently affected tests:

SlaveTest.RestartSlaveRequireExecutorAuthentication // cannot reproduce any more
ROOT_IsolatorFlags // see MESOS-8489

Issue Links

• breaks: MESOS-8391 Mesos agent doesn't notice that a pod task exits or crashes after the agent restart
• incorporates: MESOS-8489 LinuxCapabilitiesIsolatorFlagsTest.ROOT_IsolatorFlags is flaky
• is duplicated by: MESOS-7082 ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.KillTask/0 is flaky.
• is duplicated by: MESOS-8005 Mesos.SlaveTest.ShutdownUnregisteredExecutor is flaky
• is related to: MESOS-7082 ROOT_DOCKER_DockerAndMesosContainerizers/DefaultExecutorTest.KillTask/0 is flaky.
• relates to: MESOS-8005 Mesos.SlaveTest.ShutdownUnregisteredExecutor is flaky