Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-19703

Add Suppress/Revive support to the Mesos Spark Driver

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: 2.1.0
    • Fix Version/s: None
    • Component/s: Mesos
    • Labels:

      Description

      Due to the problem described here: https://issues.apache.org/jira/browse/MESOS-6112, Running > 5 Mesos frameworks concurrently can result in starvation. For example, running 10 jobs could result in 5 of them getting all the offers, even after they've launched all their executors. This leads to starvation of the other jobs. We must implement explicit SUPPRESS and REVIVE calls in the Spark Dispatcher to solve this problem.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                mgummelt Michael Gummelt
              • Votes:
                2 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: