[MESOS-5291] LinuxFilesystemIsolatorTest.ROOT_SandboxEnvironmentVariable is flaky.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Component/s: test
• Labels: isolation mesosphere

Environment

CentOS 7 without SSL

• Epic Link: flaky
• Sprint: Mesosphere Sprint 37
• Assignee: Gilbert Song
• Reporter: Gilbert Song
• Created: 27/Apr/16 03:38
• Updated: 26/Apr/17 17:01
• Resolved: 23/Jun/16 15:51

Description

Observed on the internal Mesosphere CI:

[03:05:29] :	 [Step 10/10] [ RUN      ] LinuxFilesystemIsolatorTest.ROOT_SandboxEnvironmentVariable
[03:05:30]W:	 [Step 10/10] I0427 03:05:30.880957 32158 linux.cpp:81] Making '/mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900' a shared mount
[03:05:30]W:	 [Step 10/10] I0427 03:05:30.893527 32158 linux_launcher.cpp:101] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
[03:05:30]W:	 [Step 10/10] I0427 03:05:30.893870 32176 containerizer.cpp:703] Starting container '5c2eecfd-75a4-416a-b8e9-fba620274c97' for executor 'test_executor' of framework ''
[03:05:30]W:	 [Step 10/10] I0427 03:05:30.894084 32172 provisioner.cpp:285] Provisioning image rootfs '/mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/provisioner/containers/5c2eecfd-75a4-416a-b8e9-fba620274c97/backends/copy/rootfses/0e8c0218-6551-4dea-abf5-20e9f729dcf9' for container 5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:30]W:	 [Step 10/10] I0427 03:05:30.894450 32177 copy.cpp:128] Copying layer path '/tmp/dKE6qq/test_image' to rootfs '/mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/provisioner/containers/5c2eecfd-75a4-416a-b8e9-fba620274c97/backends/copy/rootfses/0e8c0218-6551-4dea-abf5-20e9f729dcf9'
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.358980 32172 linux.cpp:355] Bind mounting work directory from '/tmp/dKE6qq/sandbox' to '/mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/provisioner/containers/5c2eecfd-75a4-416a-b8e9-fba620274c97/backends/copy/rootfses/0e8c0218-6551-4dea-abf5-20e9f729dcf9/mnt/mesos/sandbox' for container 5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.359519 32174 linux_launcher.cpp:281] Cloning child process with flags = CLONE_NEWNS
[03:05:35]W:	 [Step 10/10] + /mnt/teamcity/work/4240ba9ddd0997c3/build/src/mesos-containerizer mount --help=false --operation=make-rslave --path=/
[03:05:35]W:	 [Step 10/10] + grep -E /mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/.+ /proc/self/mountinfo
[03:05:35]W:	 [Step 10/10] + grep -v 5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:35]W:	 [Step 10/10] + cut '-d ' -f5
[03:05:35]W:	 [Step 10/10] + xargs --no-run-if-empty umount -l
[03:05:35] :	 [Step 10/10] Changing root to /mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/provisioner/containers/5c2eecfd-75a4-416a-b8e9-fba620274c97/backends/copy/rootfses/0e8c0218-6551-4dea-abf5-20e9f729dcf9
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.459079 32173 containerizer.cpp:1717] Executor for container '5c2eecfd-75a4-416a-b8e9-fba620274c97' has exited
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.459105 32173 containerizer.cpp:1481] Destroying container '5c2eecfd-75a4-416a-b8e9-fba620274c97'
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.460752 32173 cgroups.cpp:2676] Freezing cgroup /sys/fs/cgroup/freezer/mesos/5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.462360 32172 cgroups.cpp:1409] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos/5c2eecfd-75a4-416a-b8e9-fba620274c97 after 1.578752ms
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.463892 32176 cgroups.cpp:2694] Thawing cgroup /sys/fs/cgroup/freezer/mesos/5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.465405 32173 cgroups.cpp:1438] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos/5c2eecfd-75a4-416a-b8e9-fba620274c97 after 1.494272ms
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.466672 32177 linux.cpp:825] Unmounting sandbox/work directory '/mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/provisioner/containers/5c2eecfd-75a4-416a-b8e9-fba620274c97/backends/copy/rootfses/0e8c0218-6551-4dea-abf5-20e9f729dcf9/mnt/mesos/sandbox' for container 5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:35]W:	 [Step 10/10] I0427 03:05:35.466837 32176 provisioner.cpp:338] Destroying container rootfs at '/mnt/teamcity/temp/buildTmp/LinuxFilesystemIsolatorTest_ROOT_SandboxEnvironmentVariable_TU5900/provisioner/containers/5c2eecfd-75a4-416a-b8e9-fba620274c97/backends/copy/rootfses/0e8c0218-6551-4dea-abf5-20e9f729dcf9' for container 5c2eecfd-75a4-416a-b8e9-fba620274c97
[03:05:50] :	 [Step 10/10] ../../src/tests/containerizer/filesystem_isolator_tests.cpp:1386: Failure
[03:05:50] :	 [Step 10/10] Failed to wait 15secs for wait
[03:06:13] :	 [Step 10/10] [  FAILED  ] LinuxFilesystemIsolatorTest.ROOT_SandboxEnvironmentVariable (21793 ms)

Issue Links

• is related to: MESOS-4943 Reduce the size of LinuxRootfs in tests.