Details
-
Improvement
-
Status: Resolved
-
Major
-
Resolution: Won't Fix
-
0.20.2
-
None
Description
In 0.20, the fair scheduler compares all the jobs based on their running tasks, minimum slots and deficit. If the number of running tasks is less than the number of minimum slots, it will be scheduled first.
Consider a pool with minimum slot of 1000 but already have 5000 running tasks.
If we launch another job on this pool, this new job will receive minimum slots based on its weight. This new job may have higher weight if NewJobWeightBooster is used.
So this new job will still get extra slots even if the pool's running tasks are way more than the minimum slots.
The latest version does not have this problem because it first compares pool then compares jobs in the pool.