[MESOS-9081] cgroups::verify is expensive and is done implicitly during cgroups operations.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.7.0
• Component/s: containerization
• Labels: performance
• Assignee: Jie Yu
• Reporter: Benjamin Mahler
• Created: 17/Jul/18 00:32
• Updated: 20/Aug/18 23:31
• Resolved: 20/Aug/18 18:18

Description

See MESOS-8418 for how this initially came up.

Currently, many of the cgroup helper functions perform an internal verify:
https://github.com/apache/mesos/blob/1.6.0/src/linux/cgroups.cpp#L922

This reads /procs/mounts to see which cgroups subsystems are mounted, and /proc/mounts can get rather large and expensive to read.

The steady state case (polling /containers and /monitor/snapshot to retrieve container resource usage statistics) was addressed with a short term patch in MESOS-8418. However, we should consider some longer-term fixes that address performance of other events that incur cgroup operations (e.g. updating resources of a container, launching a container, etc):

1. Consider moving the verify function to public and have the isolators use it where appropriate.
2. Complementary to 1, optimize verify (e.g. read /proc/self/mountstats as suggested in MESOS-8418).

Issue Links

• is related to: MESOS-8418 mesos-agent high cpu usage because of numerous /proc/mounts reads