[MESOS-6248] mesos-slave cannot start , Assertion `isError()' failed

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.1.0

Environment

mesos 1.01 centos 7

• Assignee: Jan Schlicht
• Reporter: kasim
• Shepherd: Michael Park
• Created: 26/Sep/16 07:14
• Updated: 27/Sep/16 01:21
• Resolved: 26/Sep/16 12:57

Description

There occurs an error

mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result<T>::error() const [with T = std::basic_string<char>; std::string = std::basic_string<char>]: Assertion `isError()' failed.

when I start `mesos-slave`.

What I do before this error happen:

• service docker stop and add a `registry-mirror`
• service docker start
• service mesos-slave start

Log as below:

Sep 26 15:29:56 zk2 mesos-slave[5050]: I0926 15:29:56.421854  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594670214119757days
Sep 26 15:30:56 zk2 mesos-slave[5050]: I0926 15:30:56.422523  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594670214119757days
Sep 26 15:31:56 zk2 mesos-slave[5050]: I0926 15:31:56.423301  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594654510807141days
Sep 26 15:31:57 zk2 mesos-slave[5050]: W0926 15:31:57.067425  5070 slave.cpp:2376] Ignoring updating pid for framework ef169d8a-24fc-41d1-8b0d-c67718937a48-0000 because it does not exist
Sep 26 15:32:56 zk2 mesos-slave[5050]: I0926 15:32:56.424700  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days
Sep 26 15:33:56 zk2 mesos-slave[5050]: I0926 15:33:56.425257  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days
Sep 26 15:34:56 zk2 mesos-slave[5050]: I0926 15:34:56.425698  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days
Sep 26 15:35:56 zk2 mesos-slave[5050]: I0926 15:35:56.426959  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days
Sep 26 15:36:56 zk2 mesos-slave[5050]: I0926 15:36:56.428061  5070 slave.cpp:4591] Current disk usage 24.36%. Max allowed age: 4.594667358972014days
Sep 26 15:37:56 zk2 mesos-slave[5050]: I0926 15:37:56.429162  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:38:56 zk2 mesos-slave[5050]: I0926 15:38:56.429811  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:39:56 zk2 mesos-slave[5050]: I0926 15:39:56.430641  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:40:56 zk2 mesos-slave[5050]: I0926 15:40:56.431457  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:41:56 zk2 mesos-slave[5050]: I0926 15:41:56.432929  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:42:56 zk2 mesos-slave[5050]: I0926 15:42:56.433922  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:43:56 zk2 mesos-slave[5050]: I0926 15:43:56.434734  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597191309582847days
Sep 26 15:44:56 zk2 mesos-slave[5050]: I0926 15:44:56.435595  5070 slave.cpp:4591] Current disk usage 24.33%. Max allowed age: 4.597189882008970days
Sep 26 15:45:48 zk2 mesos-slave[5050]: W0926 15:44:56.435595  5050 logging.cpp:91] RAW: Received signal SIGTERM from process 1 of user 0; exiting
Sep 26 15:51:42 zk2 mesos-slave[4475]: WARNING: Logging before InitGoogleLogging() is written to STDERR
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.072932  4475 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073128  4475 main.cpp:244] Version: 1.0.1
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073138  4475 main.cpp:247] Git tag: 1.0.1
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.073144  4475 main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
Sep 26 15:51:42 zk2 mesos-slave[4475]: I0926 15:51:42.075127  4475 logging.cpp:194] INFO level logging started!
Sep 26 15:51:42 zk2 mesos-slave[4475]: mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result<T>::error() const [with T = std::basic_string<char>; std::string = std::basic_string<char>]: Assertion `isError()' failed.
Sep 26 15:51:42 zk2 mesos-slave[4475]: *** Aborted at 1474876302 (unix time) try "date -d @1474876302" if you are using GNU date ***
Sep 26 15:51:42 zk2 mesos-slave[4475]: PC: @     0x7f795491e5f7 __GI_raise
Sep 26 15:51:42 zk2 mesos-slave[4475]: *** SIGABRT (@0x117b) received by PID 4475 (TID 0x7f79577b98c0) from PID 4475; stack trace: ***
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f79551d8100 (unknown)
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f795491e5f7 __GI_raise
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f795491fce8 __GI_abort
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f7954917566 __assert_fail_base
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f7954917612 __GI___assert_fail
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f7955f3be4a Result<>::error()
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f795663a9a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f795663b0bd systemd::exists()
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @           0x41063c main
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @     0x7f795490ab15 __libc_start_main
Sep 26 15:51:42 zk2 mesos-slave[4475]:    @           0x412099 (unknown)
Sep 26 15:52:02 zk2 mesos-slave[4506]: WARNING: Logging before InitGoogleLogging() is written to STDERR
Sep 26 15:52:02 zk2 mesos-slave[4506]: I0926 15:52:02.413128  4506 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
Sep 26 15:52:02 zk2 mesos-slave[4506]: I0926 15:52:02.413291  4506 main.cpp:244] Version: 1.0.1
Sep 26 15:52:02 zk2 mesos-slave[4506]: I0926 15:52:02.413301  4506 main.cpp:247] Git tag: 1.0.1
Sep 26 15:52:02 zk2 mesos-slave[4506]: I0926 15:52:02.413307  4506 main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
Sep 26 15:52:02 zk2 mesos-slave[4506]: I0926 15:52:02.414762  4506 logging.cpp:194] INFO level logging started!
Sep 26 15:52:02 zk2 mesos-slave[4506]: mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result<T>::error() const [with T = std::basic_string<char>; std::string = std::basic_string<char>]: Assertion `isError()' failed.
Sep 26 15:52:02 zk2 mesos-slave[4506]: *** Aborted at 1474876322 (unix time) try "date -d @1474876322" if you are using GNU date ***
Sep 26 15:52:02 zk2 mesos-slave[4506]: PC: @     0x7fd70eb165f7 __GI_raise
Sep 26 15:52:02 zk2 mesos-slave[4506]: *** SIGABRT (@0x119a) received by PID 4506 (TID 0x7fd7119b18c0) from PID 4506; stack trace: ***
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd70f3d0100 (unknown)
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd70eb165f7 __GI_raise
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd70eb17ce8 __GI_abort
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd70eb0f566 __assert_fail_base
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd70eb0f612 __GI___assert_fail
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd710133e4a Result<>::error()
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd7108329a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd7108330bd systemd::exists()
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @           0x41063c main
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @     0x7fd70eb02b15 __libc_start_main
Sep 26 15:52:02 zk2 mesos-slave[4506]:    @           0x412099 (unknown)
Sep 26 15:52:22 zk2 mesos-slave[4584]: WARNING: Logging before InitGoogleLogging() is written to STDERR
Sep 26 15:52:22 zk2 mesos-slave[4584]: I0926 15:52:22.970069  4584 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
Sep 26 15:52:22 zk2 mesos-slave[4584]: I0926 15:52:22.970333  4584 main.cpp:244] Version: 1.0.1
Sep 26 15:52:22 zk2 mesos-slave[4584]: I0926 15:52:22.970345  4584 main.cpp:247] Git tag: 1.0.1
Sep 26 15:52:22 zk2 mesos-slave[4584]: I0926 15:52:22.970352  4584 main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
Sep 26 15:52:22 zk2 mesos-slave[4584]: I0926 15:52:22.973122  4584 logging.cpp:194] INFO level logging started!
Sep 26 15:52:22 zk2 mesos-slave[4584]: mesos-slave: ../../3rdparty/stout/include/stout/result.hpp:134: const string& Result<T>::error() const [with T = std::basic_string<char>; std::string = std::basic_string<char>]: Assertion `isError()' failed.
Sep 26 15:52:22 zk2 mesos-slave[4584]: *** Aborted at 1474876342 (unix time) try "date -d @1474876342" if you are using GNU date ***
Sep 26 15:52:22 zk2 mesos-slave[4584]: PC: @     0x7f5348b8b5f7 __GI_raise
Sep 26 15:52:22 zk2 mesos-slave[4584]: *** SIGABRT (@0x11e8) received by PID 4584 (TID 0x7f534ba268c0) from PID 4584; stack trace: ***
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f5349445100 (unknown)
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f5348b8b5f7 __GI_raise
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f5348b8cce8 __GI_abort
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f5348b84566 __assert_fail_base
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f5348b84612 __GI___assert_fail
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f534a1a8e4a Result<>::error()
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f534a8a79a3 _ZZN7systemd6existsEvENKUlvE_clEv.isra.297
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f534a8a80bd systemd::exists()
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @           0x41063c main
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @     0x7f5348b77b15 __libc_start_main
Sep 26 15:52:22 zk2 mesos-slave[4584]:    @           0x412099 (unknown)
Sep 26 15:52:43 zk2 mesos-slave[4831]: WARNING: Logging before InitGoogleLogging() is written to STDERR
Sep 26 15:52:43 zk2 mesos-slave[4831]: I0926 15:52:43.165402  4831 main.cpp:243] Build: 2016-08-26 23:06:27 by centos
Sep 26 15:52:43 zk2 mesos-slave[4831]: I0926 15:52:43.165585  4831 main.cpp:244] Version: 1.0.1
Sep 26 15:52:43 zk2 mesos-slave[4831]: I0926 15:52:43.165598  4831 main.cpp:247] Git tag: 1.0.1
Sep 26 15:52:43 zk2 mesos-slave[4831]: I0926 15:52:43.165604  4831 main.cpp:251] Git SHA: 3611eb0b7eea8d144e9b2e840e0ba16f2f659ee3
Sep 26 15:52:43 zk2 mesos-slave[4831]: I0926 15:52:43.167140  4831 logging.cpp:194] INFO level logging started!

Issue Links

• relates to: MESOS-6253 Make sure the 'Result' errors are checked correctly.