[MESOS-8259] IOSwitchboardTest.ContainerAttach is flaky.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: test
• Labels: flaky-test

Environment

Ubuntu 16.04

• Assignee: Unassigned
• Reporter: Alex R
• Created: 23/Nov/17 12:07
• Updated: 25/Jan/18 18:11
• Resolved: 25/Jan/18 18:11

Description

Observed it today in out CI:

[ RUN      ] IOSwitchboardTest.ContainerAttach
I1123 01:28:29.068318  4542 containerizer.cpp:301] Using isolation { environment_secret, network/cni, filesystem/posix, posix/cpu }
I1123 01:28:29.069078  4542 provisioner.cpp:259] Using default backend 'overlay'
I1123 01:28:29.070669 17203 containerizer.cpp:668] Recovering containerizer
I1123 01:28:29.071708 17203 provisioner.cpp:455] Provisioner recovery complete
I1123 01:28:29.072427 17207 containerizer.cpp:1195] Starting container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f
I1123 01:28:29.072916 17207 containerizer.cpp:1367] Checkpointed ContainerConfig at '/tmp/IOSwitchboardTest_ContainerAttach_oVtobZ/containers/203c6c53-7ca0-46d7-b4b7-28948bcfe44f/config'
I1123 01:28:29.072935 17207 containerizer.cpp:2919] Transitioning the state of container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f from PROVISIONING to PREPARING
I1123 01:28:29.074801 17207 switchboard.cpp:416] Allocated pseudo terminal '/dev/pts/0' for container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f
I1123 01:28:29.075064 17207 switchboard.cpp:545] Launching 'mesos-io-switchboard' with flags '--heartbeat_interval="30secs" --help="false" --socket_address="/tmp/mesos-io-switchboard-ba0fb5bc-4942-4cb4-bd20-6235a3bcf0f8" --stderr_from_fd="7" --stderr_to_fd="2" --stdin_to_fd="7" --stdout_from_fd="7" --stdout_to_fd="1" --tty="true" --wait_for_connection="false"' for container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f
I1123 01:28:29.078411 17207 switchboard.cpp:575] Created I/O switchboard server (pid: 17758) listening on socket file '/tmp/mesos-io-switchboard-ba0fb5bc-4942-4cb4-bd20-6235a3bcf0f8' for container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f
I1123 01:28:29.080448 17208 containerizer.cpp:1836] Launching 'mesos-containerizer' with flags '--help="false" --launch_info="{"command":{"shell":true,"value":"sleep 1000"},"environment":{"variables":[{"name":"MESOS_SANDBOX","type":"VALUE","value":"\/tmp\/IOSwitchboardTest_ContainerAttach_CDGOU4"}]},"task_environment":{},"tty_slave_path":"\/dev\/pts\/0","working_directory":"\/tmp\/IOSwitchboardTest_ContainerAttach_CDGOU4"}" --pipe_read="7" --pipe_write="9" --runtime_directory="/tmp/IOSwitchboardTest_ContainerAttach_oVtobZ/containers/203c6c53-7ca0-46d7-b4b7-28948bcfe44f" --unshare_namespace_mnt="false"'
I1123 01:28:29.083534 17208 launcher.cpp:140] Forked child with pid '17759' for container '203c6c53-7ca0-46d7-b4b7-28948bcfe44f'
I1123 01:28:29.084763 17208 containerizer.cpp:2919] Transitioning the state of container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f from PREPARING to ISOLATING
I1123 01:28:29.092840 17204 containerizer.cpp:2919] Transitioning the state of container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f from ISOLATING to FETCHING
I1123 01:28:29.093013 17204 fetcher.cpp:379] Starting to fetch URIs for container: 203c6c53-7ca0-46d7-b4b7-28948bcfe44f, directory: /tmp/IOSwitchboardTest_ContainerAttach_CDGOU4
I1123 01:28:29.093564 17204 containerizer.cpp:2919] Transitioning the state of container 203c6c53-7ca0-46d7-b4b7-28948bcfe44f from FETCHING to RUNNING
/home/ubuntu/workspace/mesos/Mesos_CI-build/FLAG/CMake/label/mesos-ec2-ubuntu-16.04/mesos/src/tests/containerizer/io_switchboard_tests.cpp:755: Failure
(connection).failure(): Failed to connect to /tmp/mesos-io-switchboard-ba0fb5bc-4942-4cb4-bd20-6235a3bcf0f8: Connection refused
[  FAILED  ] IOSwitchboardTest.ContainerAttach (194 ms)

Issue Links

• duplicates: MESOS-7742 Race conditions in IOSwitchboard: listening on unix socket and premature closing of the connection.