[MESOS-1718] Command executor can overcommit the agent.

• Type: Improvement
• Status: Accepted
• Priority: Critical
• Resolution: Unresolved
• Component/s: agent
• Labels: foundations
• Sprint: Twitter Mesos Q4 Sprint 3, Twitter Mesos Q4 Sprint 4, Twitter Mesos Q4 Sprint 5, Twitter Mesos Q4 Sprint 6
• Story Points: 3
• Assignee: Unassigned
• Reporter: Benjamin Mahler
• Shepherd: Benjamin Mahler
• Created: 18/Aug/14 23:50
• Updated: 20/Feb/19 18:35

Description

Currently we give a small amount of resources to the command executor, in addition to resources used by the command task:

https://github.com/apache/mesos/blob/0.20.0-rc1/src/slave/slave.cpp#L2448

ExecutorInfo Slave::getExecutorInfo(
    const FrameworkID& frameworkId,
    const TaskInfo& task)
{
  ...
    // Add an allowance for the command executor. This does lead to a
    // small overcommit of resources.
    executor.mutable_resources()->MergeFrom(
        Resources::parse(
          "cpus:" + stringify(DEFAULT_EXECUTOR_CPUS) + ";" +
          "mem:" + stringify(DEFAULT_EXECUTOR_MEM.megabytes())).get());
  ...
}

This leads to an overcommit of the slave. Ideally, for command tasks we can "transfer" all of the task resources to the executor at the slave / isolation level.

Issue Links

• blocks: MESOS-1466 Race between executor exited event and launch task can cause overcommit of resources
• blocks: MESOS-1807 Disallow executors with cpu only or memory only resources
• is depended upon by: MESOS-2845 Command tasks lead to a mixing of revocable / non-revocable cpus and memory within the container.
• relates to: MESOS-3892 Add a helper function to the Agent to retrieve the list of executors that are using optimistically offered, revocable resources.