[MESOS-8626] The 'allocatable' check in the allocator is problematic with multi-role frameworks

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Resolved
• Affects Version/s: 1.3.2, 1.4.1, 1.5.0
• Fix Version/s: 1.4.2, 1.5.2, 1.6.1, 1.7.0
• Component/s: allocation
• Labels: mesosphere multitenancy
• Target Version/s: 1.7.0
• Sprint: Mesosphere Sprint 2018-21, Mesosphere Sprint 2018-22, Mesosphere Sprint 2018-23
• Story Points: 3
• Assignee: Benjamin Bannier
• Reporter: Jie Yu
• Shepherd: Jie Yu
• Created: 02/Mar/18 00:49
• Updated: 02/Jul/18 23:26
• Resolved: 02/Jul/18 23:26

Description

The allocatable check in the allocator (shown below) was originally introduced to help alleviate the situation where a framework receives only disk, but not cpu/memory, thus cannot launch a task.

bool HierarchicalAllocatorProcess::allocatable(
    const Resources& resources)
{
  Option<double> cpus = resources.cpus();
  Option<Bytes> mem = resources.mem();

  return (cpus.isSome() && cpus.get() >= MIN_CPUS) ||
         (mem.isSome() && mem.get() >= MIN_MEM);
}

When we introduce multi-role capability to the frameworks, this check makes less sense now. For instance, consider the following case:
1) There is a single agent and a single framework in the cluster
2) The agent has cpu/memory reserved to role A, and disk reserved to B
3) The framework subscribes to both role A and role B
4) The framework expects that it'll receive an offer containing the resources on the agent
5) However, the framework receives no disk resources due to the following code. This is counter intuitive.

void HierarchicalAllocatorProcess::__allocate()
{
  ...
  Resources resources = available.allocatableTo(role);
  if (!allocatable(resources)) {
    break;
  }
  ...
}

bool Resources::isAllocatableTo(
    const Resource& resource,
    const std::string& role)
{
  CHECK(!resource.has_role()) << resource;
  CHECK(!resource.has_reservation()) << resource;

  return isUnreserved(resource) ||
         role == reservationRole(resource) ||
         roles::isStrictSubroleOf(role, reservationRole(resource));
}

Two comments:
1) If `allocatable` check is still necessary (see MESOS-7398)?
2) If we want to keep `allocatable` check for the original purpose, we should do that based on framework not role, given that a framework can subscribe to multiple roles now?

Some related JIRAs:
MESOS-1688
MESOS-7398

Issue Links

• is blocked by: MESOS-8935 Quota limit "chopping" can lead to cpu-only and memory-only offers.
• is related to: MESOS-1688 No offers if no memory is allocatable
• is related to: MESOS-7398 HierarchicalAllocatorProcess::allocatable makes strong assumptions about both resource providers and users.
• is superceded by: MESOS-8935 Quota limit "chopping" can lead to cpu-only and memory-only offers.
• relates to: MESOS-6844 Avoid offer fragmentation between multiple frameworks / within a single framework.