Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-5192

LinuxFilesystemIsolatorTest.ROOT_MultipleContainers is flaky

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Environment:

      CentOS 7 + SSL

    • Epic Link:
    • Sprint:
      Mesosphere Sprint 37

      Description

      Observed on internal Mesosphere CI:

      [11:32:03] :	 [Step 11/11] [ RUN      ] LinuxFilesystemIsolatorTest.ROOT_MultipleContainers
      [11:32:09]W:	 [Step 11/11] I0412 11:32:09.587877 17436 linux.cpp:81] Making '/tmp/NMWl31' a shared mount
      [11:32:09]W:	 [Step 11/11] I0412 11:32:09.603153 17436 linux_launcher.cpp:101] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
      [11:32:09]W:	 [Step 11/11] I0412 11:32:09.604372 17456 containerizer.cpp:682] Starting container 'f1f5de4c-aaef-45c7-b28c-83014327eb40' for executor 'test_executor1' of framework ''
      [11:32:09]W:	 [Step 11/11] I0412 11:32:09.604940 17450 provisioner.cpp:285] Provisioning image rootfs '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53' for container f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:09]W:	 [Step 11/11] I0412 11:32:09.605561 17454 copy.cpp:128] Copying layer path '/tmp/NMWl31/test_image1' to rootfs '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53'
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.784813 17450 linux.cpp:355] Bind mounting work directory from '/tmp/NMWl31/slaves/test_slave/frameworks/executors/test_executor1/runs/f1f5de4c-aaef-45c7-b28c-83014327eb40' to '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53/mnt/mesos/sandbox' for container f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.785079 17450 linux.cpp:683] Changing the ownership of the persistent volume at '/tmp/NMWl31/volumes/roles/test_role/persistent_volume_id' with uid 0 and gid 0
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.790043 17450 linux.cpp:723] Mounting '/tmp/NMWl31/volumes/roles/test_role/persistent_volume_id' to '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53/mnt/mesos/sandbox/volume' for persistent volume disk(test_role)[persistent_volume_id:volume]:32 of container f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.791724 17455 linux_launcher.cpp:281] Cloning child process with flags = CLONE_NEWNS
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.797466 17453 containerizer.cpp:682] Starting container '2c9bd571-7804-481d-9a42-1df65489dda8' for executor 'test_executor2' of framework ''
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.797947 17453 containerizer.cpp:1439] Destroying container 'f1f5de4c-aaef-45c7-b28c-83014327eb40'
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.798017 17457 provisioner.cpp:285] Provisioning image rootfs '/tmp/NMWl31/provisioner/containers/2c9bd571-7804-481d-9a42-1df65489dda8/backends/copy/rootfses/3da54ab5-8871-4913-a9d1-ae658286023a' for container 2c9bd571-7804-481d-9a42-1df65489dda8
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.798683 17451 copy.cpp:128] Copying layer path '/tmp/NMWl31/test_image2' to rootfs '/tmp/NMWl31/provisioner/containers/2c9bd571-7804-481d-9a42-1df65489dda8/backends/copy/rootfses/3da54ab5-8871-4913-a9d1-ae658286023a'
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.802534 17456 cgroups.cpp:2676] Freezing cgroup /sys/fs/cgroup/freezer/mesos/f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.804713 17451 cgroups.cpp:1409] Successfully froze cgroup /sys/fs/cgroup/freezer/mesos/f1f5de4c-aaef-45c7-b28c-83014327eb40 after 2.123008ms
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.807065 17450 cgroups.cpp:2694] Thawing cgroup /sys/fs/cgroup/freezer/mesos/f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.809074 17450 cgroups.cpp:1438] Successfully thawed cgroup /sys/fs/cgroup/freezer/mesos/f1f5de4c-aaef-45c7-b28c-83014327eb40 after 1.96992ms
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.984756 17457 containerizer.cpp:1674] Executor for container 'f1f5de4c-aaef-45c7-b28c-83014327eb40' has exited
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.987437 17451 linux.cpp:798] Unmounting volume '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53/mnt/mesos/sandbox/volume' for container f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.987499 17451 linux.cpp:817] Unmounting sandbox/work directory '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53/mnt/mesos/sandbox' for container f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:32:13]W:	 [Step 11/11] I0412 11:32:13.987915 17455 provisioner.cpp:338] Destroying container rootfs at '/tmp/NMWl31/provisioner/containers/f1f5de4c-aaef-45c7-b28c-83014327eb40/backends/copy/rootfses/1dcaba0b-23ab-462f-be91-cd49090c4f53' for container f1f5de4c-aaef-45c7-b28c-83014327eb40
      [11:33:13] :	 [Step 11/11] ../../src/tests/containerizer/filesystem_isolator_tests.cpp:1324: Failure
      [11:33:13] :	 [Step 11/11] Failed to wait 1mins for wait1
      [11:33:19] :	 [Step 11/11] [  FAILED  ] LinuxFilesystemIsolatorTest.ROOT_MultipleContainers (76539 ms)
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gilbert Gilbert Song
                Reporter:
                neilc Neil Conway
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: