[MESOS-4604] ROOT_DOCKER_DockerHealthyTask is flaky.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Component/s: test
• Labels: flaky-test health-check mesosphere test

Environment

CentOS 6/7, Ubuntu 15.04 on AWS.

• Epic Link: flaky
• Story Points: 2
• Assignee: Joseph Wu
• Reporter: Jan Schlicht
• Created: 05/Feb/16 09:24
• Updated: 26/Nov/18 12:48
• Resolved: 26/Nov/18 12:48

Description

Log from Teamcity that is running sudo ./bin/mesos-tests.sh on AWS EC2 instances:

[18:27:14][Step 8/8] [----------] 8 tests from HealthCheckTest
[18:27:14][Step 8/8] [ RUN      ] HealthCheckTest.HealthyTask
[18:27:17][Step 8/8] [       OK ] HealthCheckTest.HealthyTask (2222 ms)
[18:27:17][Step 8/8] [ RUN      ] HealthCheckTest.ROOT_DOCKER_DockerHealthyTask
[18:27:36][Step 8/8] ../../src/tests/health_check_tests.cpp:388: Failure
[18:27:36][Step 8/8] Failed to wait 15secs for termination
[18:27:36][Step 8/8] F0204 18:27:35.981302 23085 logging.cpp:64] RAW: Pure virtual method called
[18:27:36][Step 8/8]     @     0x7f7077055e1c  google::LogMessage::Fail()
[18:27:36][Step 8/8]     @     0x7f707705ba6f  google::RawLog__()
[18:27:36][Step 8/8]     @     0x7f70760f76c9  __cxa_pure_virtual
[18:27:36][Step 8/8]     @           0xa9423c  mesos::internal::tests::Cluster::Slaves::shutdown()
[18:27:36][Step 8/8]     @          0x1074e45  mesos::internal::tests::MesosTest::ShutdownSlaves()
[18:27:36][Step 8/8]     @          0x1074de4  mesos::internal::tests::MesosTest::Shutdown()
[18:27:36][Step 8/8]     @          0x1070ec7  mesos::internal::tests::MesosTest::TearDown()
[18:27:36][Step 8/8]     @          0x16eb7b2  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
[18:27:36][Step 8/8]     @          0x16e61a9  testing::internal::HandleExceptionsInMethodIfSupported<>()
[18:27:36][Step 8/8]     @          0x16c56aa  testing::Test::Run()
[18:27:36][Step 8/8]     @          0x16c5e89  testing::TestInfo::Run()
[18:27:36][Step 8/8]     @          0x16c650a  testing::TestCase::Run()
[18:27:36][Step 8/8]     @          0x16cd1f6  testing::internal::UnitTestImpl::RunAllTests()
[18:27:36][Step 8/8]     @          0x16ec513  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
[18:27:36][Step 8/8]     @          0x16e6df1  testing::internal::HandleExceptionsInMethodIfSupported<>()
[18:27:36][Step 8/8]     @          0x16cbe26  testing::UnitTest::Run()
[18:27:36][Step 8/8]     @           0xe54c84  RUN_ALL_TESTS()
[18:27:36][Step 8/8]     @           0xe54867  main
[18:27:36][Step 8/8]     @     0x7f7071560a40  (unknown)
[18:27:36][Step 8/8]     @           0x9b52d9  _start
[18:27:36][Step 8/8] Aborted (core dumped)
[18:27:36][Step 8/8] Process exited with code 134

Happens with Ubuntu 15.04, CentOS 6, CentOS 7 quite often.

Issue Links

• is blocked by: MESOS-4633 Tests will dereference stack allocated agent objects upon assertion/expectation failure.
• is related to: MESOS-2017 Segfault with "Pure virtual method called" when tests fail