[MESOS-6200] Hope mesos support soft and hard cpu/memory resource in the task

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 0.28.2
• Component/s: containerization, docker, scheduler api
• Labels: resource-management

Environment

CentOS 7
Kernel 3.10.0-327.28.3.el7.x86_64
Mesos 0.28.2
Docker 1.11.2

• Assignee: Unassigned
• Reporter: Lei Xu
• Created: 18/Sep/16 08:09
• Updated: 31/Jul/19 21:53
• Resolved: 30/Jul/19 22:25

Description

The Docker executor maybe could support soft/hard resource limit to enable more flexible resources sharing among the applications.

	CPU	Memory
hard limit	--cpu-period & --cpu-quota	--memory & --memory-swap
soft limit	--cpu-shares	--memory-reservation

And now the task protobuf message has only one resource struct that used to describe the cgroup limit, and the docker executor handle is like the following, only --memory and --cpu-shares were set:

  if (resources.isSome()) {
    // TODO(yifan): Support other resources (e.g. disk).
    Option<double> cpus = resources.get().cpus();
    if (cpus.isSome()) {
      uint64_t cpuShare =
        std::max((uint64_t) (CPU_SHARES_PER_CPU * cpus.get()), MIN_CPU_SHARES);
      argv.push_back("--cpu-shares");
      argv.push_back(stringify(cpuShare));
    }

    Option<Bytes> mem = resources.get().mem();
    if (mem.isSome()) {
      Bytes memLimit = std::max(mem.get(), MIN_MEMORY);
      argv.push_back("--memory");
      argv.push_back(stringify(memLimit.bytes()));
    }
  }

I hope that the executor and the protobuf message could separate the resource to the two parts: soft and hard. Then the user could set 2 levels resource limits for the docker.

Issue Links

• is duplicated by: MESOS-9916 Support per-container cpu / memory bursting.
• is related to: MESOS-6426 Add rlimit support to Mesos containerizer
• relates to: MESOS-6134 Port CFS quota support to Docker Containerizer using command executor.
• relates to: MESOS-9634 Soft CPU limit for windows JobObject