[MAPREDUCE-3530] Sometimes NODE_UPDATE to the scheduler throws an NPE causing the scheduling to stop

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.23.1
• Fix Version/s: 0.23.1
• Component/s: mrv2, resourcemanager, scheduler
• Target Version/s: 0.23.1
• Hadoop Flags: Reviewed
• Release Note: Fixed an NPE occuring during scheduling in the ResourceManager.
• Assignee: Arun Murthy
• Reporter: Karam Singh
• Created: 12/Dec/11 05:36
• Updated: 05/Mar/12 02:49
• Resolved: 14/Dec/11 21:24

Description

Sometimes NODE_UPDATE to the scheduler throws NPE causes scheduling to stop but ResourceManager keeps on running.
I have been observing intermitently for last 3 weeks.
But with latest svn code. I tried to run sort twice and both times Job got stuck due to NPE.

java.lang.NullPointerException
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApp.containerLaunchedOnNode(SchedulerApp.java:181)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.containerLaunchedOnNode(CapacityScheduler.java:596)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.nodeUpdate(CapacityScheduler.java:539)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:617)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.handle(CapacityScheduler.java:77)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:294)
        at java.lang.Thread.run(Thread.java:619)