[YARN-8807] FairScheduler crashes RM with oversubscription turned on if an application is killed.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: YARN-1011
• Fix Version/s: YARN-1011
• Component/s: fairscheduler, resourcemanager
• Target Version/s: YARN-1011
• Hadoop Flags: Reviewed
• Assignee: Haibo Chen
• Reporter: Haibo Chen
• Created: 20/Sep/18 17:07
• Updated: 09/Oct/18 21:38
• Resolved: 09/Oct/18 21:17

Description

When an application, that has got opportunistic containers allocated, is killed, its containers are not released immediately.

Fair scheduler would therefore continue to try to promote such orphaned containers, which results in NPE.

java.lang.NullPointerException
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptToAssignReservedResourcesOrPromoteOpportunisticContainers(FairScheduler.java:1158)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.attemptScheduling(FairScheduler.java:1129)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:1001)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1275)
    at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testKillingApplicationWithOpportunisticContainersAssigned(TestFairScheduler.java:4019)