[MESOS-2176] Hierarchical allocator inconsistently accounts for reserved resources.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.22.0
• Component/s: allocation
• Sprint: Twitter Mesos Q4 Sprint 5
• Story Points: 5
• Assignee: Benjamin Mahler
• Reporter: Benjamin Mahler
• Shepherd: Jie Yu
• Created: 05/Dec/14 19:36
• Updated: 12/Dec/14 22:00
• Resolved: 12/Dec/14 22:00

Description

Looking through the allocator code for MESOS-2099, I see an issue with respect to accounting reserved resources in the sorters:

Within HierarchicalAllocatorProcess::allocate, only unreserved resources are accounted for in the sorters, whereas everywhere else (add/remove framework, add/remove slave) we account for both reserved and unreserved.

From git blame, it looks like this issue was introduced over a long course of refactoring and fixes to the allocator. My guess is that this was never caught due to the lack of unit-testability of the allocator (unnecessarily requires a master PID to use an allocator).

From my understanding, the two levels of the hierarchical sorter should have the following semantics:

1. Level 1 sorts across roles. Only unreserved resources are shared across roles, and therefore the "role sorter" for level 1 should only account for the unreserved resource pool.
2. Level 2 sorts across frameworks, within a role. Both unreserved and reserved resources are shared across frameworks within a role, and therefore the "framework sorters" for level 2 should each account for the reserved resource pool for the role, as well as the unreserved resources allocated inside the role.

Issue Links

• blocks: MESOS-2099 Support acquiring/releasing resources with DiskInfo in allocator.