Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-5674

Port mapping isolator may fail in 'isolate' method.

    XMLWordPrintableJSON

    Details

      Description

      Port mapping isolator may return failure in isolate method, if a symlink to the network namespace handle using that ContainerId already existed. We should overwrite the symlink if it exist.

      This affects a couple test failures:

      PortMappingIsolatorTest.ROOT_TooManyContainers
      PortMappingIsolatorTest.ROOT_ContainerARPExternal
      PortMappingIsolatorTest.ROOT_ContainerCMPInternal
      PortMappingIsolatorTest.ROOT_NC_HostToContainerTCP
      

      Here is an example failure test log:

      [00:28:37] :	 [Step 10/10] [ RUN      ] PortMappingIsolatorTest.ROOT_TooManyContainers
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.046444 24846 port_mapping_tests.cpp:229] Using eth0 as the public interface
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.046728 24846 port_mapping_tests.cpp:237] Using lo as the loopback interface
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.058758 24846 resources.cpp:572] Parsing resources as JSON failed: cpus:2;mem:1024;disk:1024;ephemeral_ports:[30001-30999];ports:[31000-32000]
      [00:28:37]W:	 [Step 10/10] Trying semicolon-delimited string format instead
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.059711 24846 port_mapping.cpp:1557] Using eth0 as the public interface
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.059998 24846 port_mapping.cpp:1582] Using lo as the loopback interface
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061126 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh3 = '1024'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061172 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh1 = '128'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061206 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_wmem = '4096	16384	4194304'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061256 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_synack_retries = '5'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061297 24846 port_mapping.cpp:1869] /proc/sys/net/core/rmem_max = '212992'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061331 24846 port_mapping.cpp:1869] /proc/sys/net/core/somaxconn = '128'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061360 24846 port_mapping.cpp:1869] /proc/sys/net/core/wmem_max = '212992'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061390 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_rmem = '4096	87380	6291456'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061419 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_time = '7200'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061450 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/neigh/default/gc_thresh2 = '512'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061480 24846 port_mapping.cpp:1869] /proc/sys/net/core/netdev_max_backlog = '1000'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061511 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_intvl = '75'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061540 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_keepalive_probes = '9'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061569 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_max_syn_backlog = '512'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.061599 24846 port_mapping.cpp:1869] /proc/sys/net/ipv4/tcp_retries2 = '15'
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.069964 24846 linux_launcher.cpp:101] Using /sys/fs/cgroup/freezer as the freezer hierarchy for the Linux launcher
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.070144 24846 resources.cpp:572] Parsing resources as JSON failed: ports:[31000-31499]
      [00:28:37]W:	 [Step 10/10] Trying semicolon-delimited string format instead
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.070677 24867 port_mapping.cpp:2512] Using non-ephemeral ports {[31000,31500)} and ephemeral ports [30208,30720) for container container1 of executor ''
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.071688 24846 linux_launcher.cpp:281] Cloning child process with flags = CLONE_NEWNS | CLONE_NEWNET
      [00:28:37]W:	 [Step 10/10] I0606 00:28:37.084079 24863 port_mapping.cpp:2576] Bind mounted '/proc/11997/ns/net' to '/run/netns/11997' for container container1
      [00:28:37] :	 [Step 10/10] ../../src/tests/containerizer/port_mapping_tests.cpp:1438: Failure
      [00:28:37] :	 [Step 10/10] (isolator.get()->isolate(containerId1, pid.get())).failure(): Failed to symlink the network namespace handle '/var/run/mesos/netns/container1' -> '/run/netns/11997': File exists
      [00:28:37] :	 [Step 10/10] [  FAILED  ] PortMappingIsolatorTest.ROOT_TooManyContainers (57 ms)
      

        Attachments

          Activity

            People

            • Assignee:
              gilbert Gilbert Song
              Reporter:
              gilbert Gilbert Song
              Shepherd:
              Jie Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: