[FLINK-4904] Add a limit for how much data may be spilled in checkpoint alignments

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.1.3
• Component/s: Runtime / State Backends
• Assignee: Stephan Ewen
• Reporter: Stephan Ewen
• Created: 25/Oct/16 08:51
• Updated: 02/Dec/16 14:33
• Resolved: 02/Dec/16 14:33

Description

This should act as a safety net in cases where the sources or the network misbehave or cannot start a checkpoint. The alignment phases would then simply abort after a while, signaling the checkpoint to be incomplete.

This needs the following changes

• Add "cancel barriers" that are sent to downstream tasks
• Barrier Buffer and Barrier Tracker need to handle the "cancel barriers"
• Tasks need to react to an aborted checkpoint by responding with an appropriate message

Issue Links

• duplicates: FLINK-4975 Add a limit for how much data may be buffered during checkpoint alignment