Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Duplicate
-
1.11.3, 1.12.2, 1.13.0
-
None
-
None
Description
Currently a BLOCKING result partition is consumable only when all the partitions of the corresponding IntermediateResult become consumable. This is an unnecessary complication and has caused some problems (FLINK-21707).
Therefore I propose to simplify it to let BLOCKING result partitions be consumable individually.
Note that this will result in the scheduling to become execution-vertex-wise instead of stage-wise, with a nice side effect towards better resource utilization. The PipelinedRegionSchedulingStrategy can be simplified along with change to get rid of the correlatedResultPartitions.
Attachments
Issue Links
- duplicates
-
FLINK-22017 Regions may never be scheduled when there are cross-region blocking edges
- Closed
- is related to
-
FLINK-21707 Job is possible to hang when restarting a FINISHED task with POINTWISE BLOCKING consumers
- Closed