Details
-
New Feature
-
Status: Resolved
-
Major
-
Resolution: Incomplete
-
2.1.0
-
None
Description
Due to the problem described here: https://issues.apache.org/jira/browse/MESOS-6112, Running > 5 Mesos frameworks concurrently can result in starvation. For example, running 10 jobs could result in 5 of them getting all the offers, even after they've launched all their executors. This leads to starvation of the other jobs. We must implement explicit SUPPRESS and REVIVE calls in the Spark Dispatcher to solve this problem.
Attachments
Issue Links
- is duplicated by
-
SPARK-20447 spark mesos scheduler suppress call
- Resolved
- relates to
-
SPARK-19702 Increasse refuse_seconds timeout in the Mesos Spark Dispatcher
- Resolved
-
MESOS-6112 Frameworks are starved when > 5 are run concurrently
- Resolved