[MESOS-9334] Container stuck at ISOLATING state due to libevent poll never returns.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.4.3, 1.5.3, 1.6.2, 1.7.1, 1.8.0
• Component/s: containerization
• Sprint: Mesosphere RI-6 Sprint 2018-31, Containerization R7 Sprint 32
• Story Points: 5
• Assignee: Qian Zhang
• Reporter: Qian Zhang
• Shepherd: Gilbert Song
• Created: 19/Oct/18 10:02
• Updated: 16/Jan/19 00:45
• Resolved: 30/Oct/18 23:04

Description

We found UCR container may be stuck at `ISOLATING` state:

2018-10-03 09:13:23: I1003 09:13:23.274561 2355 containerizer.cpp:3122] Transitioning the state of container 1e5b8fc3-5c9e-4159-a0b9-3d46595a5b54 from PREPARING to ISOLATING
2018-10-03 09:13:23: I1003 09:13:23.279223 2354 cni.cpp:962] Bind mounted '/proc/5244/ns/net' to '/run/mesos/isolators/network/cni/1e5b8fc3-5c9e-4159-a0b9-3d46595a5b54/ns' for container 1e5b8fc3-5c9e-4159-a0b9-3d46595a5b54
2018-10-03 09:23:22: I1003 09:23:22.879868 2354 containerizer.cpp:2459] Destroying container 1e5b8fc3-5c9e-4159-a0b9-3d46595a5b54 in ISOLATING state

 In the above logs, the state of container `1e5b8fc3-5c9e-4159-a0b9-3d46595a5b54` was transitioned to `ISOLATING` at 09:13:23, but did not transitioned to any other states until it was destroyed due to the executor registration timeout (10 mins). And the destroy can never complete since it needs to wait for the container to finish isolating.

Issue Links

• is duplicated by: MESOS-9320 UCR container launch stuck at PROVISIONING during image fetching.
• is related to: MESOS-9320 UCR container launch stuck at PROVISIONING during image fetching.