[MESOS-2018] Dynamic Reservation

• Type: Epic
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: agent, allocation, framework, master
• Labels: mesosphere offer persistence reservations resource stateful storage
• Epic Name: Dynamic Reservation
• Sprint: Mesosphere Q4 Sprint 2 - 11/14, Mesosphere Q4 Sprint 3 - 12/7, Mesosphere Q1 Sprint 1 - 1/23, Mesosphere Q1 Sprint 2 - 2/6, Mesosphere Q1 Sprint 3 - 2/20
• Assignee: Michael Park
• Reporter: Adam B
• Shepherd: Adam B
• Created: 31/Oct/14 00:54
• Updated: 26/May/17 21:27
• Resolved: 10/Mar/17 20:28

Description

Overview

This is a feature to provide better support for running stateful services on Mesos such as HDFS (Distributed Filesystem), Cassandra (Distributed Database), or MySQL (Local Database).
Current resource reservations (henceforth called "static" reservations) are statically determined by the slave operator at slave start time, and individual frameworks have no authority to reserve resources themselves.
Dynamic reservations allow a framework to dynamically reserve offered resources, such that those resources will only be re-offered to the same framework (or other frameworks with the same role).
This is especially useful if the framework's task stored some state on the slave, and needs a guaranteed set of resources reserved so that it can re-launch a task on the same slave to recover that state.

Planned Stages

1. MESOS-2489: Enable a framework to perform reservation operations.

The goal of this stage is to allow the framework to send back a Reserve/Unreserve operation which gets validated by the master and updates the allocator resources. The allocator's allocate logic is left unchanged and the resources get offered back to the framework's role as desired.

2. MESOS-2491: Persist the reservation state on the slave.

The goal of this stage is to persist the reservation state on the slave. Currently the master knows to store the persistent volumes in the checkpointedResources data structure which gets sent to individual slaves to be checkpointed. We will update the master such that dynamically reserved resources are stored in the checkpointedResources as well. This stage also involves subtasks such as updating the slave re(register) logic to support slave re-starts.

3. MESOS-2600: Introduce reservation HTTP endpoints on the master.

The goal of this stage is to enable operators to perform reservation operations via HTTP endpoints on the master.

Issue Links

• blocks: MESOS-7577 Remove GPU_RESOURCES capability and master flag `--filter-gpu-resources={true|false}`
• blocks: MESOS-7579 Deprecate GPU_RESOURCES capability and master flag `--filter-gpu-resources={true|false}`
• blocks: MESOS-2727 0.23.0 Release
• is related to: MESOS-1554 Persistent resources support for storage-like services
• relates to: MESOS-7229 Dynamic Reservation Improvements
• relates to: MESOS-1791 Introduce Master / Offer Resource Reservations aka Quota.
• relates to: MESOS-2101 Add the persistent resources release primitive to the framework API
• relates to: MESOS-7576 Add master flag `--filter-gpu-resources={true|false}`