Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Fixed
-
None
-
Foundations: RI-20 59, Studio 4: RI-21 60, Studio 4: RI-21 61
-
13
Description
Currently, the v1 operator API has poor performance relative to the v0 json API. The following initial numbers were provided by Will Mahler from our state serving benchmark:
| OPTIMIZED - Master (baseline) | ||||
| Test setup | 1000 agents with a total of 10000 running tasks and 10000 completed tasks | 10000 agents with a total of 100000 running tasks and 100000 completed tasks | 20000 agents with a total of 200000 running tasks and 200000 completed tasks | 40000 agents with a total of 400000 running tasks and 400000 completed tasks |
| v0 'state' response | 0.17 | 1.66 | 8.96 | 12.42 |
| v1 x-protobuf | 0.35 | 3.21 | 9.47 | 19.09 |
| v1 json | 0.45 | 4.72 | 10.81 | 31.43 |
There is quite a lot of variance, but v1 protobuf consistently slower than v0 (sometimes significantly so) and v1 json is consistently slower than v1 protobuf (sometimes significantly so).
The reason that the v1 operator API is slower is that it does the following:
(1) Construct temporary unversioned state response object by copying in-memory un-versioned state into overall response object. (expensive!)
(2) Evolve it to v1: serialize, de-serialize into v1 overall state object. (expensive!)
(3) Serialize the overall v1 state object to protobuf or json.
(4) Destruct the temporaries (expensive! but is done after response starts serving)
On the other hand, the v0 jsonify approach does the following:
(1) Serialize the in-memory unversioned state into json, by traversing state and accumulating the overall serialized json.
This means that v1 has substantial overhead vs v0, and we need to remove it to bring v1 on-par or better than v0. v1 should serialize directly to json (straightforward with jsonify) or protobuf (this can be done via a io::CodedOutputStream).