[FLINK-4815] Automatic fallback to earlier checkpoints when checkpoint restore fails

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: Runtime / Checkpointing
• Labels: auto-deprioritized-major
• Assignee: Unassigned
• Reporter: Stephan Ewen
• Created: 12/Oct/16 18:29
• Updated: 25/Jun/21 22:38

Description

Flink should keep multiple completed checkpoints.
When the restore of one completed checkpoint fails for a certain number of times, the CheckpointCoordinator should fall back to an earlier checkpoint to restore.

Issue Links

• is blocked by: FLINK-4714 Set task state to RUNNING after state has been restored
• is blocked by: FLINK-4754 Make number of retained checkpoints user configurable
• is blocked by: FLINK-17012 Expose stage of task initialization

Sub-Tasks

• Executions failed from "DEPLOYING" should retain restored checkpoint information: Open
• Checkpoint Coordinator should be called to restore state with a specific checkpoint ID: Open
• RestartStrategy should track how many failed restore attempts the same checkpoint has and fall back to earlier checkpoints: Closed
• Don't always remove checkpoints in ZooKeeperCompletedCheckpointStore#recover(): Closed