[NEMO-53] Make SchedulingPolicy Stackable

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.1
• Labels: pull-request-available
• Assignee: Jae Hyeon Park
• Reporter: JangHo Seo
• Created: 10/May/18 05:42
• Updated: 29/Nov/18 11:14
• Resolved: 15/May/18 08:00

Description

Current runtime allows only one SchedulingPolicy at a time. It has multiple drawbacks.

First, the executional behavior of RoundRobinSchedulingPolicy is quite overloaded:

• It filters out executors which are saturated with active TaskGroups.
• It filters out executors with inapproporate ContianerType (Transient, Reserved, Compute, ...).
• For the remaining executors, it chooses one executor in round-robin fashion.
• For TaskGroup lifecycle events, it reacts by signaling to the SchedulerRunner and the corresponding ExecutorRepresenters.

Second, it is extremely hard to add additional 'filtering rule' flexibly.

To add another filtering predicate, one should edit RoundRobinSchedulingPolicy to incorporate the additional predicate, or create another SchedulingPolicy and make it use RoundRobinSchedulingPolicy as a hard-wired fallback.

It makes sense to allow user-configurable stack of multiple SchedulingPolicy, each of which defines filtering predicate.

Issue Links

• blocks: NEMO-73 SchedulingPolicy as Vertex-level Execution Property
• links to: GitHub Pull Request #15