Details
-
Improvement
-
Status: Accepted
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Of interest to improving scalability is the ability for the master to quickly fan out offers as the allocator makes new allocation decisions.
Some low hanging fruit:
- Low hanging copy elimination.
- Send more compact and cache-friendlier data structures across master <-> allocator interface boundary (right now the allocator makes a call back into the master with a double-nested hashmap<role,hashmap<SlaveID,Resources>>).
- Reduce excessive map lookups.
Longer term:
- Update v1 message evolution paths to avoid de-serializing / serializing in favor of copying fields (either manually or with a reflection based copier), or by moving individual fields when possible (this will only help with string fields AFAICT).
- Update the master <-> allocator interface to send all framework allocations in one call, and use a parallel loop (see MESOS-8587) in the master to speed up the fan-out.
Attachments
Issue Links
- relates to
-
MESOS-8587 Introduce a parallel for each loop (and other parallel algorithms).
- Accepted