[YARN-5279] Potential Container leak in NM in preemption flow

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Implemented
• Component/s: nodemanager, resourcemanager
• Assignee: Rohith Sharma K S
• Reporter: Rohith Sharma K S
• Created: 21/Jun/16 10:57
• Updated: 03/Nov/16 14:50
• Resolved: 03/Nov/16 14:50

Description

In discussion YARN-4862 comment, it is observed that there could be a container leak in NodeManager whenever container is preempted from RM

Basically if NM receives same containerId details in containersToCleanUp and containersToBeRemovedFromNM in the same heartbeat then container will never-ever removed in NMContext. Rather NM kills the container of containersToCleanup and send back status again to RM. But RM blindly reject the status since RMContainer is already removed and it is null.

I think whenever RMContainer is null, RMNode should be informed to send containersToBeRemovedFromNM so that NM will remove from its context.