Details
Major Description
Currently, there is no sibling communication between tasks - this implies that a task can be completed by the first vertex in a wave of tasks, but the entire wave of tasks has to complete before success can be reported.
This occurs in limit + filter query patterns common between the data access engines.
select * from data where x > 1 limit 10;
will run through a full-table scan worth of tasks to generate 10 rows per task, to aggregate it to produce the final 10 row result.
The VertexManager receives counters/events early enough to short-circuit the rest of the vertex tasks, to prevent the remainder of tasks from getting scheduled when the limit condition has been satisfied by an initial sub-set of the tasks.
This is a specialization of the VertexManagerPlugin for this common case scheduling pattern.