[MESOS-6112] Frameworks are starved when > 5 are run concurrently

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.0.1
• Component/s: allocation, master
• Assignee: Unassigned
• Reporter: Michael Gummelt
• Created: 31/Aug/16 00:21
• Updated: 09/May/18 00:56
• Resolved: 09/May/18 00:56

Description

As I understand it, the master will send an offer to a list of frameworks ordered by DRF, until the offer is accepted. There is a 1s wait time between each offering. Once the decline timeout for the first framework has been reached, rather than continuing to submit the offer to the rest of the frameworks in the list, the master starts over at the beginning, starving the rest of the frameworks.

This means that in order for Mesos to support > 5 concurrent frameworks, all frameworks must be good citizens and set their decline timeout to something large or suppress offers. I think this is a fairly undesirable state of things.

I propose that the master instead continues to submit the offer to every registered framework, even if the declineOffer timeout has been reached.

The potential increase in task startup latency that could be introduced by this change can be obviated in part if we also make the master smarter about how long to wait between successive offers, rather than a static 1s.

Issue Links

• duplicates: MESOS-3202 Avoid role/framework offer starvation in DRF allocator.
• is related to: SPARK-19703 Add Suppress/Revive support to the Mesos Spark Driver
• relates to: MESOS-3202 Avoid role/framework offer starvation in DRF allocator.
• relates to: SPARK-20483 Mesos Coarse mode may starve other Mesos frameworks if max cores is not a multiple of executor cores
• relates to: MESOS-6111 Offer cycle is undocumented