[YARN-1572] Low chance to hit NPE issue in AppSchedulingInfo#allocateNodeLocal

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.3.0
• Component/s: scheduler
• Assignee: Wenwu Peng
• Reporter: Wenwu Peng
• Created: 08/Jan/14 09:07
• Updated: 07/Jan/17 01:52

Description

we have lower chance to hit NPE in allocateNodeLocal when run benchmark(hit 4 in 20 times).

2014-07-31 04:18:19,653 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerNode: Assigned container container_1406794589275_0001_01_000021 of capacity <memory:1024, vCores:1> on host datanode10:57281, which has 6 containers, <memory:6144, vCores:6> used and <memory:2048, vCores:2> available after allocation
2014-07-31 04:18:19,654 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in handling event type NODE_UPDATE to the scheduler
java.lang.NullPointerException
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocateNodeLocal(AppSchedulingInfo.java:311)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.AppSchedulingInfo.allocate(AppSchedulingInfo.java:268)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.FiCaSchedulerApp.allocate(FiCaSchedulerApp.java:136)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainer(FifoScheduler.java:683)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignNodeLocalContainers(FifoScheduler.java:602)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainersOnNode(FifoScheduler.java:560)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.assignContainers(FifoScheduler.java:488)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.nodeUpdate(FifoScheduler.java:729)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:774)
        at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fifo.FifoScheduler.handle(FifoScheduler.java:101)
        at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:599)
        at java.lang.Thread.run(Thread.java:662)
2014-07-31 04:18:19,655 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Exiting, bbye..

Issue Links

• is related to: YARN-1663 NPE in FifoScheduler.assignContainers