Details
-
Sub-task
-
Status: Closed
-
Major
-
Resolution: Fixed
-
llap
-
None
-
None
Description
Wait queue ordering is currently based on the following
1) Finishable or not
2) Vertex parallelism
3) Attempt start time
This ordering has issues in cases when 2 or more queries are execute in parallel, then all map tasks gets scheduled first even though reduce task of some dag are in finishable state. This will make the first submitted query slower even though it can proceed to completion.
Add fair ordering to wait queue comparator to take dag submission time into account. For the above scenario, if we take dag submission time into account, the first submitted task will proceed to completion.