Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-6707

Mesos Scheduler should allow the user to specify constraints based on slave attributes

    XMLWordPrintableJSON

Details

    Description

      Currently, the mesos scheduler only looks at the 'cpu' and 'mem' resources when trying to determine the usablility of a resource offer from a mesos slave node. It may be preferable for the user to be able to ensure that the spark jobs are only started on a certain set of nodes (based on attributes).

      For example, If the user sets a property, let's say

      spark.mesos.constraints

      is set to

      tachyon=true;us-east-1=false

      , then the resource offers will be checked to see if they meet both these constraints and only then will be accepted to start new executors.

      Attachments

        Activity

          People

            ankurcha Ankur Chauhan
            ankurcha Ankur Chauhan
            Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: