[YARN-10475] Scale RM-NM heartbeat interval based on node utilization

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.10.1, 3.4.0
• Fix Version/s: 3.2.2, 3.4.0, 3.3.1, 3.2.3
• Component/s: yarn
• Assignee: Jim Brennan
• Reporter: Jim Brennan
• Created: 27/Oct/20 21:25
• Updated: 10/Jun/21 08:15
• Resolved: 02/Nov/20 19:46

Description

Add the ability to scale the RM-NM heartbeat interval based on node cpu utilization compared to overall cluster cpu utilization.  If a node is over-utilized compared to the rest of the cluster, it's heartbeat interval slows down.  If it is under-utilized compared to the rest of the cluster, it's heartbeat interval speeds up.

This is a feature we have been running with internally in production for several years.  It was developed by nroberts, based on the observation that larger faster nodes on our cluster were under-utilized compared to smaller slower nodes. 

This feature is dependent on YARN-10450, which added cluster-wide utilization metrics.