Details
-
Story
-
Status: Resolved
-
Major
-
Resolution: Duplicate
-
None
-
None
-
None
Description
The k8s pods abstraction is not the only way to run multiple containers on a single host. In my use of Mesos, I have two use cases for running multiple containers together:
1. I run a job that needs a "babysitter" process to run with it, such that it can inspect the filesystem being written to by the primary job. The babysitter verifies that the job is operating correctly, and takes automated recovery/termination actions on failure.
2. I run a webserver that writes logs. I also run logstash in the same container, in order to ship the logs to our main log storage system.
In both of these cases, I need to run common initialization code, and then I start the various sub-tasks that need to be isolated. Rather than need my framework's scheduler to be able to make decisions at launch time about how it wants to carve up the process, I'd like my executor to be able to locally create subgroups for CPU and Memory containerization, so that it can decide on-the-fly whether to carve its container into smaller pieces.
I think that by triggering the splitting of containers in the executor, this will enable more dynamic usage.
The downside to this is that the container splitting would have to be rebuilt for every containerizer; the upside is the flexibility in creating temporary subcontainers only when they're needed.
I think that this would only require for command-line utilities that understood the Mesos environment (that the executor could shell out to), or perhaps it would be done with an executor API call to the slave, to ensure the subcontainers didn't break cleanup.
I've heard rumors that Aurora can do something like this; perhaps their implementation could graduate to Mesos proper?
Attachments
Issue Links
- is duplicated by
-
MESOS-2449 Support group of tasks (Pod) constructs and API in Mesos.
-
- Resolved
-