[YARN-3793] Several NPEs when deleting local files on NM recovery

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.6.0
• Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
• Component/s: nodemanager
• Target Version/s: 2.7.2
• Hadoop Flags: Reviewed
• Assignee: Varun Saxena
• Reporter: Karthik Kambatla
• Created: 10/Jun/15 20:52
• Updated: 06/Jan/17 07:27
• Resolved: 01/Jul/15 21:16

Description

When NM work-preserving restart is enabled, we see several NPEs on recovery. These seem to correspond to sub-directories that need to be deleted. I wonder if null pointers here mean incorrect tracking of these resources and a potential leak. This JIRA is to investigate and fix anything required.

Logs show:

2015-05-18 07:06:10,225 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : null
2015-05-18 07:06:10,224 ERROR org.apache.hadoop.yarn.server.nodemanager.DeletionService: Exception during execution of task in DeletionService
java.lang.NullPointerException
        at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:274)
        at org.apache.hadoop.fs.FileContext.delete(FileContext.java:755)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.deleteAsUser(DefaultContainerExecutor.java:458)
        at org.apache.hadoop.yarn.server.nodemanager.DeletionService$FileDeletionTask.run(DeletionService.java:293)