[MESOS-5673] Port mapping isolator may cause segfault if it bind mount root does not exist.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.28.2
• Fix Version/s: 0.28.3, 1.0.0
• Component/s: containerization
• Labels: isolator mesosphere networking tests

Environment

Fedora 23 with network isolation

• Epic Link: Container Network Isolation
• Sprint: Mesosphere Sprint 37
• Story Points: 3
• Assignee: Gilbert Song
• Reporter: Gilbert Song
• Shepherd: Jie Yu
• Created: 21/Jun/16 02:02
• Updated: 26/Apr/17 16:53
• Resolved: 22/Jun/16 23:02

Description

A check is needed for port mapping isolator for its bind mount root. Otherwise, non-existed port-mapping bind mount root may cause segmentation fault for some cases. Here is the test log:

[00:57:42] :	 [Step 10/10] [----------] 11 tests from PortMappingIsolatorTest
[00:57:42] :	 [Step 10/10] [ RUN      ] PortMappingIsolatorTest.ROOT_NC_ContainerToContainerTCP
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.723029 24841 port_mapping_tests.cpp:229] Using eth0 as the public interface
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.723348 24841 port_mapping_tests.cpp:237] Using lo as the loopback interface
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.735090 24841 resources.cpp:572] Parsing resources as JSON failed: cpus:2;mem:1024;disk:1024;ephemeral_ports:[30001-30999];ports:[31000-32000]
[00:57:42]W:	 [Step 10/10] Trying semicolon-delimited string format instead
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.736006 24841 port_mapping.cpp:1557] Using eth0 as the public interface
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.736331 24841 port_mapping.cpp:1582] Using lo as the loopback interface
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737501 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh3 = '1024'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737545 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh1 = '128'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737578 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_wmem = '4096	16384	4194304'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737608 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_synack_retries = '5'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737637 24841 port_mapping.cpp:1869] /proc/sys/net/core/rmem_max = '212992'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737666 24841 port_mapping.cpp:1869] /proc/sys/net/core/somaxconn = '128'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737694 24841 port_mapping.cpp:1869] /proc/sys/net/core/wmem_max = '212992'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737720 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_rmem = '4096	87380	6291456'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737746 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_time = '7200'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737772 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh2 = '512'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737798 24841 port_mapping.cpp:1869] /proc/sys/net/core/netdev_max_backlog = '1000'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737828 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_intvl = '75'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737854 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_probes = '9'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737879 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_max_syn_backlog = '512'
[00:57:42]W:	 [Step 10/10] I0604 00:57:42.737905 24841 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_retries2 = '15'
[00:57:42]W:	 [Step 10/10] F0604 00:57:42.737968 24841 port_mapping_tests.cpp:448] CHECK_SOME(isolator): Failed to get realpath for bind mount root '/var/run/netns': Not found 
[00:57:42]W:	 [Step 10/10] *** Check failure stack trace: ***
[00:57:42]W:	 [Step 10/10]     @     0x7f8bd52583d2  google::LogMessage::Fail()
[00:57:42]W:	 [Step 10/10]     @     0x7f8bd525832b  google::LogMessage::SendToLog()
[00:57:42]W:	 [Step 10/10]     @     0x7f8bd5257d21  google::LogMessage::Flush()
[00:57:42]W:	 [Step 10/10]     @     0x7f8bd525ab92  google::LogMessageFatal::~LogMessageFatal()
[00:57:42]W:	 [Step 10/10]     @           0xa62171  _CheckFatal::~_CheckFatal()
[00:57:42]W:	 [Step 10/10]     @          0x1931b17  mesos::internal::tests::PortMappingIsolatorTest_ROOT_NC_ContainerToContainerTCP_Test::TestBody()
[00:57:42]W:	 [Step 10/10]     @          0x19e17b6  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
[00:57:42]W:	 [Step 10/10]     @          0x19dc864  testing::internal::HandleExceptionsInMethodIfSupported<>()
[00:57:42]W:	 [Step 10/10]     @          0x19bd2ae  testing::Test::Run()
[00:57:42]W:	 [Step 10/10]     @          0x19bda66  testing::TestInfo::Run()
[00:57:42]W:	 [Step 10/10]     @          0x19be0b7  testing::TestCase::Run()
[00:57:42]W:	 [Step 10/10]     @          0x19c4bf5  testing::internal::UnitTestImpl::RunAllTests()
[00:57:42]W:	 [Step 10/10]     @          0x19e247d  testing::internal::HandleSehExceptionsInMethodIfSupported<>()
[00:57:42]W:	 [Step 10/10]     @          0x19dd3a4  testing::internal::HandleExceptionsInMethodIfSupported<>()
[00:57:42]W:	 [Step 10/10]     @          0x19c38d1  testing::UnitTest::Run()
[00:57:42]W:	 [Step 10/10]     @           0xfd28cb  RUN_ALL_TESTS()
[00:57:42]W:	 [Step 10/10]     @           0xfd24b1  main
[00:57:42]W:	 [Step 10/10]     @     0x7f8bceb89580  __libc_start_main
[00:57:42]W:	 [Step 10/10]     @           0xa607c9  _start
[00:57:43]W:	 [Step 10/10] /mnt/teamcity/temp/agentTmp/custom_script659125926639545396: line 3: 24841 Aborted                 (core dumped) GLOG_v=1 ./bin/mesos-tests.sh --verbose --gtest_filter="$GTEST_FILTER"
[00:57:43]W:	 [Step 10/10] Process exited with code 134