[MESOS-8952] process::await/collect n^2 performance issue

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.4.2, 1.5.2, 1.6.1, 1.7.0
• Component/s: libprocess
• Labels: performance
• Target Version/s: 1.4.2, 1.5.2, 1.6.1
• Assignee: Benjamin Mahler
• Reporter: Benjamin Mahler
• Created: 24/May/18 21:31
• Updated: 12/Jul/18 22:30
• Resolved: 24/May/18 21:51

Description

Due to the use of std::list::size (which appears to be linear in complexity even with g++ and c++11), process::await and process::collect suffer from n^2 complexity. A minimal patch to switch to std::vector shows the following improvement:

Registered 2000 frameworks
Finished launching the tasks; Sleep 10 seconds ...
Start collecting metrics ...
v0 '/metrics/snapshot' response took 17.751689014secs
v1 'master::call::GetMetrics' application/x-protobuf response took 17.523928635secs
v1 'master::call::GetMetrics' application/json response took 18.111901732secs

Registered 2000 frameworks
Finished launching the tasks; Sleep 10 seconds ...
Start collecting metrics ...
v0 '/metrics/snapshot' response took 1.730948431secs
v1 'master::call::GetMetrics' application/x-protobuf response took 1.697177667secs
v1 'master::call::GetMetrics' application/json response took 2.160314525secs

A follow up to switch the interface to std::vector would be beneficial since we don't need any of the std::list benefits.

Issue Links

• is related to: MESOS-9072 Improve performance of metrics snapshot.