[MESOS-4235] JSON generation performance improvement

• Type: Epic
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: libprocess, master, stout
• Labels: mesosphere scalability twitter
• Epic Name: json-perf
• Assignee: Michael Park
• Reporter: Michael Park
• Shepherd: Benjamin Hindman
• Created: 22/Dec/15 00:28
• Updated: 02/May/17 23:05
• Resolved: 04/Apr/17 18:49

Description

This is an epic which evolved from MESOS-2353. As mentioned in the description of MESOS-2353, most of the work is spent performing memory allocation/deallocation. Some preliminary efforts have been made such as calling reserve for JSON::Array. There are still plenty of dynamic allocations being made especially from instances of JSON::Object which hold a std::map as a member.

The current approach being adopted is to introduce a jsonify function which by-passes these unnecessary dynamic allocations and copying, and to simply hold references to the underlying objects.

We plan to first introduce the jsonify function to stout, and update master's state endpoint, then proceed to update the rest of the system.

Issue Links

• incorporates: MESOS-2353 Improve performance of the state.json endpoint for large clusters.
• is related to: MESOS-701 Improve webui performance for large clusters.