[MESOS-9801] Use protobuf arenas for v1 API responses.

• Type: Improvement
• Status: Accepted
• Priority: Major
• Resolution: Unresolved
• Component/s: agent, master
• Labels: performance
• Target Version/s: 1.9.0
• Assignee: William Mahler
• Reporter: Benjamin Mahler
• Created: 28/May/19 19:39
• Updated: 28/May/19 19:41

Description

The v1 API response construction is currently slower than the v0 API response construction. A primary reason for this is that the v1 API constructs intermediate C++ protobuf response objects, which are very expensive in terms of memory allocation/deallocation cost. Also involved is the use of evolve() which evolves messages from unversioned protobuf into v1 protobuf. This also has very high memory allocation / deallocation cost.

Using arenas for all v1 API response construction will provide a significant improvement.

This ticket currently captures all the aspects of this:

• Updating evolve() to use arenas across all v1 API responses.
• Updating all response construction functions (e.g. getState())) to use arenas.
• Making this change for both the master and agent.

This is blocked by MESOS-9755 since we need to upgrade our bundled protobuf to have string fields allocated in the arenas.

We may split out tickets for CHANGELOG purposes if only a portion of this lands in 1.9.0.

Issue Links

• is blocked by: MESOS-9755 Upgrade bundled protobuf to 3.7.x.