[FLINK-21066] Refactor the CheckpointCoordinator to compute the tasks to trigger/wait/commit dynamically

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.13.0
• Component/s: API / DataStream, Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Yun Gao
• Reporter: Yun Gao
• Created: 21/Jan/21 10:16
• Updated: 28/May/21 08:57
• Resolved: 06/Feb/21 02:57

Description

Currently Flink would identify the tasks to trigger/wait/commit in compile phase, and pass them to the CheckpointCoordinator via JobGraph and cache them in CheckpointCoordinator. However, to support checkpoint after some tasks finished, we might need to recompute the tasks to trigger/wait/commit for each checkpoint, thus we would first refactor CheckpointCoordinator to compute the tasks to trigger/wait/commit dynamically, but keeps the computing logic unchange (e.g., tasks to trigger are source tasks and tasks to wait/commit are all tasks)

Issue Links

• links to: GitHub Pull Request #14734