[YARN-10087] ATS possible NPE on REST API when data is missing

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: ATSv2
• Labels: newbie
• Assignee: Samrat Deb
• Reporter: Wilfred Spiegelenburg
• Created: 16/Jan/20 03:09
• Updated: 12/Jul/22 18:32

Description

If the data stored by the ATS is not complete REST calls to the ATS can return a NPE instead of results.

{{

{"exception":"NullPointerException","javaClassName":"java.lang.NullPointerException"}

}}

The issue shows up when the ATS was down for a short period and in that time new applications were started. This causes certain parts of the application data to be missing in the ATS store. In most cases this is not a problem and data will be returned but when you start filtering data the filtering fails throwing the NPE.
In this case the request was for: http://<ATS-IP>:8188/ws/v1/applicationhistory/apps?user=hive'

If certain pieces of data are missing the ATS should not even consider returning that data, filtered or not. We should not display partial or incomplete data.
In case of the missing user information ACL checks cannot be correctly performed and we could see more issues.

A similar issue was fixed in YARN-7118 where the queue details were missing. It just skips the app to prevent the NPE but that is not the correct thing when the user is missing