Details
-
Task
-
Status: Resolved
-
Minor
-
Resolution: Fixed
-
None
Description
As per the discussion with jieyu and avinash.mesos , I am going to allow nested containers in TaskGroups to have separate namespaces. I am also going to retain the existing functionality, where nested containers can share namespaces with the parent/root container.
Use case: At Yelp, we have this application called seagull that runs multiple tasks in parallel. It is mainly used for running tests that depend on other containerized internal microservices. It was developed before mesos had support for docker-executor. So, it uses a custom executor, which directly talks to docker daemon on the host and run a bunch of service containers along with the process where tests are executed. Resources for all these containers are not accounted for in mesos. Clean-up of these containers is also a headache. We have a tool called docker-reaper that automatically reaps the orphaned containers once the executor goes away. In addition to that, we also run a few cron jobs that clean-up any leftover containers.
We are in the process of containerizing the process that runs the tests. We also want to delegate the responsibility of lifecycle management of docker containers to mesos and get rid of the custom executor. We looked at a few alternatives to do this and decided to go with pods because they provide all-or-nothing(atomicity) semantics that we need for our application. But, we cannot use pods directly because all the containers in a pod have the same network namespace. The service discovery mechanism requires all the containers to have separate IPs. All of our microservices bind to 8888 container port, so we will have port collision unless we are giving separate namespaces to all the containers in a pod.
Proposal: I am planning to allow nested containers to have separate namespaces. If NetworkInfo protobuf for nested containers is not empty, then we will assign separate mnt and network namespaces to the nested containers. Otherwise, they will share the network and mount namepsaces with the parent/root container.
Attachments
Issue Links
- links to
