[YARN-11360] Add number of decommissioning/shutdown nodes to YARN cluster metrics.

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0, 3.3.6
• Fix Version/s: 3.4.0, 3.2.5, 3.3.6
• Component/s: client, resourcemanager
• Labels: pull-request-available
• Target Version/s: 3.4.0, 3.2.5, 3.3.9
• Hadoop Flags: Reviewed
• Assignee: Chris Nauroth
• Reporter: Chris Nauroth
• Created: 21/Oct/22 17:48
• Updated: 25/Jan/24 15:01
• Resolved: 28/Oct/22 18:38

Description

YARN cluster metrics expose counts of NodeManagers in various states including active and decommissioned. However, these metrics don't expose NodeManagers that are currently in the process of decommissioning. This can look a little spooky to a consumer of these metrics. First, the node drops out of the active count, so it seems like a node just vanished. Then, later (possibly hours later with consideration of graceful decommission), it comes back into existence in the decommissioned count.

This issue tracks adding the decommissioning count to the metrics ResourceManager RPC. This also enables exposing it in the yarn top output. This metric is already visible through the REST API, so there isn't any change required there.

Issue Links

• links to: GitHub Pull Request #5060