[MESOS-9053] Network ports isolator can falsely trigger while destroying containers.

• Type: Bug
• Status: Accepted
• Priority: Critical
• Resolution: Unresolved
• Component/s: containerization
• Assignee: Xudong Ni
• Reporter: Xudong Ni
• Shepherd: James Peach
• Created: 06/Jul/18 18:25
• Updated: 06/Feb/19 22:39

Description

During the container destroying process, the network port resource will be cleared first and then container is destroyed, there is race condition window between port clearance and container destroyed that the network port isolator will kick in check the port isolation, the isolator will think it's port violation since the port resource is already cleared.In the following case, the race condition window is about 2.2 seconds.
 
Sample case:

I0630 06:36:19.029884 2609728 ports.cpp:533] Updated ports to [31001-31002] for container e5cf47bb-4c21-4897-a899-573e6ac37258
I0630 06:36:28.240780 2609719 ports.cpp:533] Updated ports to [31001-31002] for container e5cf47bb-4c21-4897-a899-573e6ac37258
I0630 06:43:48.280997 2609731 ports.cpp:533] Updated ports to [] for container e5cf47bb-4c21-4897-a899-573e6ac37258
I0630 06:43:48.281141 2609756 containerizer.cpp:2408] Destroying container e5cf47bb-4c21-4897-a899-573e6ac37258 in RUNNING state
I0630 06:43:48.380264 2609756 ports.cpp:601] Container e5cf47bb-4c21-4897-a899-573e6ac37258 is listening on unallocated port(s): [31002-31002]
I0630 06:43:50.477228 2609717 containerizer.cpp:2861] Container e5cf47bb-4c21-4897-a899-573e6ac37258 has exited