Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3793

Several NPEs when deleting local files on NM recovery

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: nodemanager
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When NM work-preserving restart is enabled, we see several NPEs on recovery. These seem to correspond to sub-directories that need to be deleted. I wonder if null pointers here mean incorrect tracking of these resources and a potential leak. This JIRA is to investigate and fix anything required.

      Logs show:

      2015-05-18 07:06:10,225 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : null
      2015-05-18 07:06:10,224 ERROR org.apache.hadoop.yarn.server.nodemanager.DeletionService: Exception during execution of task in DeletionService
      java.lang.NullPointerException
              at org.apache.hadoop.fs.FileContext.fixRelativePart(FileContext.java:274)
              at org.apache.hadoop.fs.FileContext.delete(FileContext.java:755)
              at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.deleteAsUser(DefaultContainerExecutor.java:458)
              at org.apache.hadoop.yarn.server.nodemanager.DeletionService$FileDeletionTask.run(DeletionService.java:293)
      

        Attachments

        1. YARN-3793.01.patch
          8 kB
          Varun Saxena
        2. YARN-3793.02.patch
          5 kB
          Varun Saxena

          Activity

            People

            • Assignee:
              varun_saxena Varun Saxena
              Reporter:
              kasha Karthik Kambatla
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: