[MESOS-8345] Improve master responsiveness while serving state information.

• Type: Epic
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.8.0
• Component/s: HTTP API, master
• Labels: mesosphere performance
• Epic Name: Master state responsiveness
• Target Version/s: 1.8.0
• Assignee: Alex R
• Reporter: Benjamin Mahler
• Created: 19/Dec/17 00:37
• Updated: 21/Aug/19 17:27
• Resolved: 05/Apr/19 19:13

Description

Currently when state is requested from the master, the response is built using the master actor. This means that when the master is building an expensive state response, the master is locked and cannot process other events. This in turn can lead to higher latency on further requests to state. Previous performance improvements to JSON generation (MESOS-4235) alleviated this issue, but for large cluster with a lot of clients this can still be a problem.

It's possible to serve state outside of the master actor by streaming the state (re-using the existing streaming operator API) into another actor(s) and serving from there.

NOTE: I believe this approach will incur a small performance cost during master failover, since the master has to perform an additional copy of state that it fans out.

Issue Links

• is related to: MESOS-8731 mesos master APIs become latent
• is related to: MESOS-8163 Add an actor for handling HTTP requests for state (including streaming)
• relates to: MESOS-9092 Adopt rapidjson for improved json serialization performance.