[MESOS-8522] `prepareMounts` in Mesos containerizer is flaky.

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.5.0
• Component/s: containerization
• Labels: containerization mesosphere
• Assignee: Gilbert Song
• Reporter: Chun-Hung Hsiao
• Shepherd: Jie Yu
• Created: 01/Feb/18 02:56
• Updated: 13/May/19 14:26

Description

The prepareMount() function in src/slave/containerizer/mesos/launch.cpp sometimes fails with the following error:

Failed to prepare mounts: Failed to mark '/home/docker/containers/af78db6ebc1aff572e576b773d1378121a66bb755ed63b3278e759907e5fe7b6/shm' as slave: Invalid argument

The error message comes from https://github.com/apache/mesos/blob/1.5.x/src/slave/containerizer/mesos/launch.cpp#L#L326.

Although it does not happen frequently, it can be reproduced by running tests that need to clone mount namespaces in repetition. For example, I just reproduced the bug with the following command after 17 minutes:

sudo bin/mesos-tests.sh --gtest_filter='*ROOT_PublishResourcesRecovery' --gtest_break_on_failure --gtest_repeat=-1 --verbose

No that in this example, the test itself does not involve any docker image or docker containerizer.