Details
-
Task
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
This is an umbrella JIRA that proposes a new REEF application Vortex, which aims to ease programming of low-priority job that run on volatile(preemptable) resources.
The motivation is that today’s datacenters are often under-utilized due to high-priority tenants that need to dynamically adjust resource usage depending on load fluctuations.
Our approach to this problem is to provide a runtime for low-priority jobs to effectively cope with non-cooperative(thus low-latency) preemptions with the following goals:
- Allow high-priority jobs to reclaim resources quickly
- Allow low-priority jobs to efficiently utilize volatile resources
Attachments
1.
|
Initial project setup for Vortex |
|
Resolved | John Yang |
2.
|
Initial Vortex implementation |
|
Resolved | John Yang |
3.
|
Simple duplicate Vortex Tasklet launch |
|
Open | John Yang |
4.
|
Distinguish different types of FailedEvaluator in Vortex |
|
Open | John Yang |
5.
|
Support Vortex Tasklet(s) cancellation by user |
|
Resolved | Andrew Chung |
6.
|
Basic Vortex profiling |
|
Open | John Yang |
7.
|
Clean up Serializable in Vortex |
|
Resolved | John Yang |
8.
|
Callback features for VortexFuture |
|
Resolved | Andrew Chung |
9.
|
Extend configuration options for Vortex resources |
|
Open | Unassigned |
10.
|
Smarter VortexWorker-Tasklet scheduling |
|
Resolved | John Yang |
11.
|
Local aggregation |
|
Resolved | Andrew Chung |
12.
|
VortexRequestor creates too many threads |
|
Open | John Yang |
13.
|
Higher-level Vortex API |
|
Open | Unassigned |