[FLINK-17014] Implement PipelinedRegionSchedulingStrategy

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.11.0
• Fix Version/s: 1.11.0
• Component/s: Runtime / Coordination
• Labels: pull-request-available
• Assignee: Zhu Zhu
• Reporter: Zhu Zhu
• Created: 07/Apr/20 06:17
• Updated: 21/Apr/20 05:50
• Resolved: 21/Apr/20 05:50

Description

The PipelinedRegionSchedulingStrategy submits one pipelined region to the DefaultScheduler each time. The PipelinedRegionSchedulingStrategy must be aware of the inputs of each pipelined region. It should schedule a region if and only if all the inputs of that region become consumable.

PipelinedRegionSchedulingStrategy can implement as below:

• startScheduling() : schedule all source regions one by one.
• onPartitionConsumable(partition) : Check all the consumer regions of the notified partition, if all the inputs of a region have turned to be consumable, schedule the region
• restartTasks(tasksToRestart) : find out all regions which contain the tasks to restart, reschedule those whose inputs are all consumable

Issue Links

• links to: GitHub Pull Request #11770