[SPARK-10471] Mesos Scheduler should reject offers not matching constraints for longer period of time

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.6.0
• Component/s: Mesos, Scheduler, Spark Core
• Labels: mesos scheduler
• Target Version/s: 1.6.0
• Assignee: Felix Bechstein
• Reporter: Felix Bechstein
• Created: 07/Sep/15 08:11
• Updated: 17/May/20 17:48
• Resolved: 09/Nov/15 21:38

Description

The current implementation of mesos constraints introduced in SPARK-6707 rejects offers not matching the constraints with default refuseDuration of 5 seconds.
Even though spark will never launch tasks on these offers, mesos keeps sending the same offers again every 5 seconds making them unavailable for other frameworks.

Spark should reject those offers for a longer period of time to prevent offer starvation when running a lot of frameworks.

Issue Links

• links to: [Github] Pull Request #8639 (felixb)