[YARN-9984] FSPreemptionThread can cause NullPointerException while app is unregistered with containers running on a node

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.0.0
• Fix Version/s: 3.3.0, 3.2.2, 3.1.4
• Component/s: fairscheduler
• Hadoop Flags: Reviewed
• Assignee: Wilfred Spiegelenburg
• Reporter: Wilfred Spiegelenburg
• Created: 18/Nov/19 01:32
• Updated: 26/Feb/20 05:29
• Resolved: 19/Nov/19 08:37

Description

When an application is unregistered there is a chance that there are still containers running on a node for that application. In all cases we handle the application missing from the RM gracefully (log a message and continue) except for the FS pre-emption thread.

In case the application is removed but some containers are still linked to a node the FSPreemptionThread will crash with a NPE when it tries to retrieve the application id for the attempt:

FSAppAttempt app =
    scheduler.getSchedulerApp(container.getApplicationAttemptId());
ApplicationId appId = app.getApplicationId();

Issue Links

• is duplicated by: YARN-8470 Fair scheduler exception with SLS