[MESOS-3949] User CGroup Isolation tests fail on Centos 6.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.26.0
• Component/s: containerization
• Labels: mesosphere

Environment

CentOS 6.6, gcc 4.8.1, on vagrant libvirt, 16GB, 8 CPUs,
../configure --enable-libevent --enable-ssl

• Target Version/s: 0.26.0
• Sprint: Mesosphere Sprint 23
• Story Points: 3
• Assignee: Alexander Rojas
• Reporter: Bernd Mathiske
• Created: 18/Nov/15 17:03
• Updated: 26/Nov/18 12:35
• Resolved: 26/Nov/18 12:35

Description

UserCgroupIsolatorTest/0.ROOT_CGROUPS_UserCgroup and UserCgroupIsolatorTest/1.ROOT_CGROUPS_UserCgroup fail on CentOS 6.6 with similar output when libevent and SSL are enabled.

sudo ./bin/mesos-tests.sh --gtest_filter="UserCgroupIsolatorTest/0.ROOT_CGROUPS_UserCgroup" --verbose

[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from UserCgroupIsolatorTest/0, where TypeParam = mesos::internal::slave::CgroupsMemIsolatorProcess
userdel: user 'mesos.test.unprivileged.user' does not exist
[ RUN      ] UserCgroupIsolatorTest/0.ROOT_CGROUPS_UserCgroup
I1118 16:53:35.273717 30249 mem.cpp:605] Started listening for OOM events for container 867a829e-4a26-43f5-86e0-938bf1f47688
I1118 16:53:35.274538 30249 mem.cpp:725] Started listening on low memory pressure events for container 867a829e-4a26-43f5-86e0-938bf1f47688
I1118 16:53:35.275164 30249 mem.cpp:725] Started listening on medium memory pressure events for container 867a829e-4a26-43f5-86e0-938bf1f47688
I1118 16:53:35.275784 30249 mem.cpp:725] Started listening on critical memory pressure events for container 867a829e-4a26-43f5-86e0-938bf1f47688
I1118 16:53:35.276448 30249 mem.cpp:356] Updated 'memory.soft_limit_in_bytes' to 1GB for container 867a829e-4a26-43f5-86e0-938bf1f47688
I1118 16:53:35.277331 30249 mem.cpp:391] Updated 'memory.limit_in_bytes' to 1GB for container 867a829e-4a26-43f5-86e0-938bf1f47688
-bash: /sys/fs/cgroup/memory/mesos/867a829e-4a26-43f5-86e0-938bf1f47688/cgroup.procs: No such file or directory
mkdir: cannot create directory `/sys/fs/cgroup/memory/mesos/867a829e-4a26-43f5-86e0-938bf1f47688/user': No such file or directory
../../src/tests/containerizer/isolator_tests.cpp:1307: Failure
Value of: os::system( "su - " + UNPRIVILEGED_USERNAME + " -c 'mkdir " + path::join(flags.cgroups_hierarchy, userCgroup) + "'")
  Actual: 256
Expected: 0
-bash: /sys/fs/cgroup/memory/mesos/867a829e-4a26-43f5-86e0-938bf1f47688/user/cgroup.procs: No such file or directory
../../src/tests/containerizer/isolator_tests.cpp:1316: Failure
Value of: os::system( "su - " + UNPRIVILEGED_USERNAME + " -c 'echo $$ >" + path::join(flags.cgroups_hierarchy, userCgroup, "cgroup.procs") + "'")
  Actual: 256
Expected: 0
[  FAILED  ] UserCgroupIsolatorTest/0.ROOT_CGROUPS_UserCgroup, where TypeParam = mesos::internal::slave::CgroupsMemIsolatorProcess (149 ms)

sudo ./bin/mesos-tests.sh --gtest_filter="UserCgroupIsolatorTest/1.ROOT_CGROUPS_UserCgroup" --verbose

[==========] Running 1 test from 1 test case.
[----------] Global test environment set-up.
[----------] 1 test from UserCgroupIsolatorTest/1, where TypeParam = mesos::internal::slave::CgroupsCpushareIsolatorProcess
userdel: user 'mesos.test.unprivileged.user' does not exist
[ RUN      ] UserCgroupIsolatorTest/1.ROOT_CGROUPS_UserCgroup
I1118 17:01:00.550706 30357 cpushare.cpp:392] Updated 'cpu.shares' to 1024 (cpus 1) for container e57f4343-1a97-4b44-b347-803be47ace80
-bash: /sys/fs/cgroup/cpuacct/mesos/e57f4343-1a97-4b44-b347-803be47ace80/cgroup.procs: No such file or directory
mkdir: cannot create directory `/sys/fs/cgroup/cpuacct/mesos/e57f4343-1a97-4b44-b347-803be47ace80/user': No such file or directory
../../src/tests/containerizer/isolator_tests.cpp:1307: Failure
Value of: os::system( "su - " + UNPRIVILEGED_USERNAME + " -c 'mkdir " + path::join(flags.cgroups_hierarchy, userCgroup) + "'")
  Actual: 256
Expected: 0
-bash: /sys/fs/cgroup/cpuacct/mesos/e57f4343-1a97-4b44-b347-803be47ace80/user/cgroup.procs: No such file or directory
../../src/tests/containerizer/isolator_tests.cpp:1316: Failure
Value of: os::system( "su - " + UNPRIVILEGED_USERNAME + " -c 'echo $$ >" + path::join(flags.cgroups_hierarchy, userCgroup, "cgroup.procs") + "'")
  Actual: 256
Expected: 0
-bash: /sys/fs/cgroup/cpu/mesos/e57f4343-1a97-4b44-b347-803be47ace80/cgroup.procs: No such file or directory
mkdir: cannot create directory `/sys/fs/cgroup/cpu/mesos/e57f4343-1a97-4b44-b347-803be47ace80/user': No such file or directory
../../src/tests/containerizer/isolator_tests.cpp:1307: Failure
Value of: os::system( "su - " + UNPRIVILEGED_USERNAME + " -c 'mkdir " + path::join(flags.cgroups_hierarchy, userCgroup) + "'")
  Actual: 256
Expected: 0
-bash: /sys/fs/cgroup/cpu/mesos/e57f4343-1a97-4b44-b347-803be47ace80/user/cgroup.procs: No such file or directory
../../src/tests/containerizer/isolator_tests.cpp:1316: Failure
Value of: os::system( "su - " + UNPRIVILEGED_USERNAME + " -c 'echo $$ >" + path::join(flags.cgroups_hierarchy, userCgroup, "cgroup.procs") + "'")
  Actual: 256
Expected: 0
[  FAILED  ] UserCgroupIsolatorTest/1.ROOT_CGROUPS_UserCgroup, where TypeParam = mesos::internal::slave::CgroupsCpushareIsolatorProcess (116 ms)

Issue Links

• duplicates: MESOS-3470 UserCgroupIsolatorTest failed on CentOS 6.6
• is duplicated by: MESOS-3470 UserCgroupIsolatorTest failed on CentOS 6.6