[FLINK-22805] Dynamic configuration of Flink checkpoint interval

• Type: New Feature
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.13.1
• Component/s: Runtime / Checkpointing
• Labels: auto-deprioritized-critical auto-deprioritized-major
• Assignee: Unassigned
• Reporter: Fu Kai
• Created: 31/May/21 02:34
• Updated: 22/Aug/23 22:35

Description

Flink currently does not support dynamic configuration of checkpoint interval on the fly. It's useful for use cases like backfill/cold-start from a stream containing whole history.

 
In the cold-start phase, resources are fully utilized and the back-pressure is high for all upstream operators, causing the checkpoint timeout constantly. The real production traffic is far less than that and the provisioned resource is capable of handling it. 
 
With the dynamic checkpoint interval configuration, the cold-start process can be speeded up with less frequent checkpoint interval or even turned off. After the process is completed, the checkpoint interval can be updated to normal.

 

Issue Links

• is related to: FLINK-32514 FLIP-309: Support using larger checkpointing interval when source is processing backlog