[MESOS-4353] Limit the number of processes created by libprocess

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: libprocess
• Labels: libprocess mesosphere
• Sprint: Mesosphere Sprint 32
• Story Points: 1
• Assignee: Maged Michael
• Reporter: Qian Zhang
• Shepherd: Joris Van Remoortere
• Created: 13/Jan/16 13:17
• Updated: 26/Nov/18 12:20
• Resolved: 26/Nov/18 12:20

Description

Currently libprocess will create max(8, number of CPU cores) processes during the initialization, see https://github.com/apache/mesos/blob/0.26.0/3rdparty/libprocess/src/process.cpp#L2146 for details. This should be OK for a normal machine which has no much cores (e.g., 16, 32), but for a powerful machine which may have a large number of cores (e.g., an IBM Power machine may have 192 cores), this will cause too much worker threads which are not necessary.

And since libprocess is widely used in Mesos (master, agent, scheduler, executor), it may also cause some performance issue. For example, when user creates a Docker container via Mesos in a Mesos agent which is running on a powerful machine with 192 cores, the DockerContainerizer in Mesos agent will create a dedicated executor for the container, and there will be 192 worker threads in that executor. And if user creates 1000 Docker containers in that machine, then there will be 1000 executors, i.e., 1000 * 192 worker threads which is a large number and may thrash the OS.

Issue Links

• contains: MESOS-5914 mesos-docker-executor initialize many threads
• is duplicated by: MESOS-4592 add an environment variable to specify the thread pool size
• is related to: MESOS-4609 Subprocess should be more intelligent about setting/inheriting libprocess environment variables
• is related to: MESOS-4592 add an environment variable to specify the thread pool size