[MESOS-4633] Tests will dereference stack allocated agent objects upon assertion/expectation failure.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Labels: flaky mesosphere tech-debt test
• Epic Link: flaky
• Sprint: Mesosphere Sprint 28, Mesosphere Sprint 29, Mesosphere Sprint 30, Mesosphere Sprint 31
• Story Points: 5
• Assignee: Joseph Wu
• Reporter: Joseph Wu
• Shepherd: Michael Park
• Created: 09/Feb/16 22:28
• Updated: 26/Nov/18 12:20
• Resolved: 26/Nov/18 12:20

Description

Tests that use the StartSlave test helper are generally fragile when the test fails an assert/expect in the middle of the test. This is because the StartSlave helper takes raw pointer arguments, which may be stack-allocated.

In case of an assert failure, the test immediately exits (destroying stack allocated objects) and proceeds onto test cleanup. The test cleanup may dereference some of these destroyed objects, leading to a test crash like:

[18:27:36][Step 8/8] F0204 18:27:35.981302 23085 logging.cpp:64] RAW: Pure virtual method called
[18:27:36][Step 8/8]     @     0x7f7077055e1c  google::LogMessage::Fail()
[18:27:36][Step 8/8]     @     0x7f707705ba6f  google::RawLog__()
[18:27:36][Step 8/8]     @     0x7f70760f76c9  __cxa_pure_virtual
[18:27:36][Step 8/8]     @           0xa9423c  mesos::internal::tests::Cluster::Slaves::shutdown()
[18:27:36][Step 8/8]     @          0x1074e45  mesos::internal::tests::MesosTest::ShutdownSlaves()
[18:27:36][Step 8/8]     @          0x1074de4  mesos::internal::tests::MesosTest::Shutdown()
[18:27:36][Step 8/8]     @          0x1070ec7  mesos::internal::tests::MesosTest::TearDown()

The StartSlave helper should take shared_ptr arguments instead.
This also means that we can remove the Shutdown helper from most of these tests.

Issue Links

• blocks: MESOS-4604 ROOT_DOCKER_DockerHealthyTask is flaky.
• breaks: MESOS-5000 MasterTest.MasterLost is flaky
• is cloned by: MESOS-4634 Tests will dereference stack allocated master objects upon assertion/expectation failure.
• relates to: MESOS-4075 Continue test suite execution across crashing tests.