[MESOS-9852] Slow memory growth in master due to deferred deletion of offer filters and timers.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.5.4, 1.6.3, 1.7.3, 1.8.1, 1.9.0
• Component/s: allocation, master
• Labels: resource-management
• Target Version/s: 1.5.4, 1.6.3, 1.7.3, 1.8.1, 1.9.0
• Sprint: Resource Mgmt: RI 15 Sp 49
• Story Points: 3
• Assignee: Benjamin Mahler
• Reporter: Benjamin Mahler
• Created: 19/Jun/19 16:47
• Updated: 13/Aug/19 13:37
• Resolved: 24/Jun/19 20:40

Description

The allocator does not keep a handle to the offer filter timer, which means it cannot remove the timer overhead (in this case memory) when removing the offer filter earlier (e.g. due to revive):

https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L1338-L1352

In addition, the offer filter is allocated on the heap but not deleted until the timer fires (which might take forever!):

https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L1321
https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L1408-L1413
https://github.com/apache/mesos/blob/1.8.0/src/master/allocator/mesos/hierarchical.cpp#L2249

We'll need to try to backport this to all active release branches.