Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6707

Mesos Scheduler should allow the user to specify constraints based on slave attributes

    XMLWordPrintableJSON

    Details

    • Target Version/s:

      Description

      Currently, the mesos scheduler only looks at the 'cpu' and 'mem' resources when trying to determine the usablility of a resource offer from a mesos slave node. It may be preferable for the user to be able to ensure that the spark jobs are only started on a certain set of nodes (based on attributes).

      For example, If the user sets a property, let's say

      spark.mesos.constraints

      is set to

      tachyon=true;us-east-1=false

      , then the resource offers will be checked to see if they meet both these constraints and only then will be accepted to start new executors.

        Attachments

          Activity

            People

            • Assignee:
              ankurcha Ankur Chauhan
              Reporter:
              ankurcha Ankur Chauhan
            • Votes:
              1 Vote for this issue
              Watchers:
              11 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: