[MESOS-6765] Make the Resources wrapper "copy-on-write" to improve performance.

• Type: Improvement
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.7.1, 1.8.0
• Labels: mesosphere performance
• Target Version/s: 1.7.1
• Epic Link: Allocator performance improvement.
• Sprint: Mesosphere Sprint 2018-26, Mesosphere Sprint 2018-27, Mesosphere Sprint 2018-28, Mesosphere Sprint 2018-29
• Story Points: 5
• Assignee: Meng Zhu
• Reporter: Benjamin Mahler
• Shepherd: Benjamin Mahler
• Created: 09/Dec/16 02:07
• Updated: 28/Jan/19 23:17
• Resolved: 21/Sep/18 22:45

Description

Resources currently directly stores the underlying resource objects:

class Resources
{
  ...
  std::vector<Resource_> resources;
};

What this means is that copying of Resources (which occurs frequently) is expensive since copying a Resource object is relatively heavy-weight.

One strategy, in MESOS-4770, is to avoid protobuf in favor of C++ types (i.e. replace Value::Scalar, Value::Set, and Value::Ranges with C++ equivalents). However, metadata like reservations, disk info, etc, is still fairly expensive to copy even if avoiding protobufs.

An approach to reduce copying would be to only copy the resource objects upon writing, when there are multiple references to the resource object. If there is a single reference to the resource object we could safely mutate it without copying. E.g.

class Resource
{
  ...
  std::vector<shared_ptr<Resource_>> resources;
};

// Mutation function:
void Resources::mutate(size_t index)
{
  // Copy if there are multiple references.
  if (resources[i].use_count() > 1) {
    resources[i] = copy(resources[i]);
  }

  // Mutate safely.
  resources[i].some_mutation();
}

On the other hand, this introduces a additional level of pointer chasing. So we would need to weigh the approaches.

Issue Links

• relates to: MESOS-4770 Investigate performance improvements for 'Resources' class.