[YARN-11152] QueueMetrics is leaking memory when creating a new queue during reinitialisation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.4.0
• Fix Version/s: 3.4.0
• Component/s: capacity scheduler
• Labels: pull-request-available
• Target Version/s: 3.4.0
• Hadoop Flags: Reviewed
• Assignee: András Győri
• Reporter: András Győri
• Created: 16/May/22 07:26
• Updated: 28/Jan/24 05:24
• Resolved: 18/May/22 16:30

Description

Capacity Scheduler handles reinitialisation by reparsing the entire queue hierarchy, then reinitialising the old queue hierarchy by taking the newly parsed queues into account. After this, the newly parsed queues are discarded and they are GCed.
However, with the introduction of YARN-6492, we are storing a parent queue in QueueMetrics, which is problematic, because at that point, the parent queue could still point to a parent reference, that is a newly parsed parent queue (which should be discarded after the reinitialisation). Due to this fact, QueueMetrics could contain parents members of an entirely different queue hierarchy than the current hierarchy in use. It could lead to subtle problems as well as memory leak, because one parent reference will keep the whole queue hierarchy alive.
This problem arised when we programatically added one queue after an other via the mutation API, thus keeping alive hundreds of queue hierarchies at the same time, crippling the GC and the whole RM.

Issue Links

• links to: GitHub Pull Request #4313