Uploaded image for project: 'Aurora'
  1. Aurora
  2. AURORA-1579

Allow preflight-check of Job schedulability

    XMLWordPrintableJSON

Details

    • Task
    • Status: Open
    • Minor
    • Resolution: Unresolved
    • None
    • None
    • Client, Scheduler
    • None

    Description

      The goal of this feature is to allow users to check if their job (as configured) would likely be schedulable given Aurora's current offers. An extended form of this feature would be able to perform this test while assuming any current instance of the job in question would be stopped.

      Here is the suggestion I sent to the mailing list describing my use-case for such a feature:

      We currently run a (relatively) small Mesos/Aurora cluster, and don't always have significant resource overhead available.

      Sometimes, we go to schedule a job and we're just short of what we estimated-by-hand we'd need in the cluster for it. Most of the tasks schedule - but a few stay "PENDING" because of the resource constraint. This often confuses users, or in some cases, causes the command to block for a while until it eventually times out.

      We're currently working in-house on automating somewhat-more-precise basic estimation with information sourced from /offers to get a sense of "nope, your task won't schedule" to provide fast feedback that doesn't manipulate the state of the cluster.

      However, our basic estimation doesn't include co-scheduling constraints, quotas, etc., which seem like something Aurora would be able to determine.

      It is worth noting that this kind of feature is inherently subject to race conditions and future restrictions. Somewhat paradoxically, this feature is more useful the smaller your quota or cluster is, as many actions in a restricted environment will require adding capacity (or quota). It is worth documenting this feature to mention that there are cases where your tasks could still end up pending - losing a race, host failure, "oddly shaped tasks" failing to reschedule, etc.

      Attachments

        Activity

          People

            Unassigned Unassigned
            bmhatfield Brian Hatfield
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: