[MAPREDUCE-3862] Nodemanager can appear to hang on shutdown due to lingering DeletionService threads

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.23.1
• Fix Version/s: 0.23.2
• Component/s: mrv2, nodemanager
• Target Version/s: 0.23.2
• Assignee: Jason Darrell Lowe
• Reporter: Jason Darrell Lowe
• Created: 15/Feb/12 17:31
• Updated: 21/Feb/12 19:19
• Resolved: 17/Feb/12 23:01

Description

When a nodemanager attempts to shutdown cleanly, it's possible for it to appear to hang due to lingering DeletionService threads. This can occur when yarn.nodemanager.delete.debug-delay-sec is set to a relatively large value and one or more containers executes on the node shortly before the shutdown.

The DeletionService is never calling setExecuteExistingDelayedTasksAfterShutdownPolicy() on the ScheduledThreadPoolExecutor, and it defaults to waiting for all scheduled tasks to complete before exiting.

Issue Links

• duplicates: MAPREDUCE-3746 Nodemanagers are not automatically shut down after decommissioning