Details
-
Improvement
-
Status: Accepted
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
Description
Currently the resource estimator returns the amount of oversubscription resources that are available, since resources cannot be negative, this allows the resource estimator to express the following:
(1) Return empty resources: We are fully allocated for oversubscription resources.
(2) Return non-empty resources: We are under-allocated for oversubscription resources. In other words, some are available.
However, there is an additional situation that we cannot express:
(3) Analogous to returning non-empty "negative" resources: We are over-allocated for oversubscription resources. Do not re-offer any of the over-allocated oversubscription resources that are recovered.
Without (3), the slave can only shrink the total pool of oversubscription resources by returning (1) as resources are recovered, until the pool is shrunk to the desired size. However, this approach is only best-effort, it's possible for a framework to launch more tasks in the window of time (15 seconds by default) that the slave polls the estimator.